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(4) learning self-assessment; (5) ongoing formative assessment; (6) 
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Summary 

This publication describes the 
developmental work of a group of 
practitioners who sought to identify 
strategies for recognising learning 
progress on non-accredited 
programmes. Seven case studies 
make up the main body of the report, 
and each provides details of work 
carried out by an organisation that 
provides non-accredited learning 
opportunities for adults. In all cases, 
the centrality of the learner is of key 
importance, and on-going NIACE 
research on learners’ perspectives 
provides an important contribution to 
the debate. The final chapter identifies 
a framework for good practice and 
discusses issues that are common 
knowledge, the areas in need of 
further development and what further 
research needs to be carried out in 
the field of recognising and validating 
non-accredited adult learning. 
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Key Findings 

1 We identified nine elements in the process 
of recording and validating achievement. 
What follows is one framework of good 
practice: it is not suggested that each 
element is always necessary, nor that all 
nine are always sufficient. Where 
appropriate, we indicate how the element of 
good practice relates to the requirements of 
the Common Inspection Framework. 

■ Initial identification of learning 
objectives: tutors clearly and accurately 
identify primary learning aims in 
advance of the course. Inspection will 
consider the extent to which teachers 
and trainers ‘plan effectively with clear 
objectives that all learners understand’. 

■ Initial assessment of learners: this 
provides an opportunity to gather 
information about learners’ knowledge 
and experience, assess additional 
learning support needs, discuss and 
negotiate the learning programme, and 
adapt the programme to meet learners’ 
needs. Inspection will look at the extent 
to which ‘initial assessment provides an 
accurate basis on which to plan an 
appropriate programme of work. 
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Negotiation of learning objectives with 
learners: tutors to share aims and 
objectives with learners, and to agree on 
what is to be retained, amended, added 
to or subtracted from any initial 
aspirations. All new objectives to be 
discussed and agreed with students. 
Inspection will evaluate the extent to 
which tutors ‘with learners, develop 
individual learning plans, informed by 
initial assessment, that are reviewed 
and updated regularly’. 

Learner self-assessment: learners to be 
encouraged to develop an 
understanding of what is going well or 
not so well, and how they might improve 
on progress. It is now recognised that 
‘learning to learn’ is an important feature 
of the learning process, and that the 
recognition and evaluation of learning 
progress plays an important part in this. 
Inspection will examine the extent to 
which learners ‘develop the skills of 
critical evaluation’ and ‘understand how 
well they are progressing and what they 
need to do to improve’. 

On-going formative assessment: 
formative assessment is integral to the 
process of recording achievement. 
‘Assessment’ in this context implies no 
more (and no less) than a discerning 
judgement about learner’s progress; it is 
‘on-going’ in the sense that it goes on all 
the time; and it is formative in so far as 
its purpose is forward-looking, aiming to 
improve future learning (as distinct from 
the retrospective nature of summative 
assessment). Inspection will look at the 
extent to which ‘assessment is used to 
monitor progress and inform individual 
learners about how they are performing 
and how they might develop further’. 



■ Progression advice and guidance: it is a 
recognised feature of good practice to 
provide advice and guidance to students 
about further courses, learning 
programmes, and employment and 
other opportunities that are related to 
the course they have completed. 
Inspection will look at the extent to 
which ‘careers education and guidance 
are effective in guiding learners towards 
the opportunities available to them when 
they have completed their studies or 
training’. 

■ Record of achievement: a record of 
achievement is in essence an 
assessment of learners’ progress as 
judged against their learning aims and 
objectives. The record should refer to 
clearly identified learning objectives, and 
take account of classroom observation, 
learner self-assessment and tutor 
assessment. Inspection will look at the 
extent to which ‘achievements towards 
learning goals are recorded’ and also at 
the extent to which ‘assessment 
information is used to guide course and 
programme development’. 

■ Moderation of assessment: internal 
moderation enables validation of tutor 
assessment. Tutors make presentations 
of samples of assessed work to 
colleagues together with a Head of 
Department or Curriculum Manager. The 
purpose of internal moderation is to 
discuss difficult cases, and identify 
common standards and criteria for 
assessment. 

■ Celebration of achievement: in many 
cases, nothing is more important for the 
endorsement of learning than an event 
at which learners’ success is celebrated. 



2 We identified several areas in which there is 
room for further development. Notable 
amongst these is the area of staff 
development, and in particular: identifying 
aims and objectives; negotiating learning 
objectives; facilitating learner self- 
assessment; formative assessment; 
describing learning outcomes. 

3 Our research confirmed that we remain 
without adequate answers to four key 
questions: 

■ How do learners benefit from the 
process of recording and validating 
achievement? 

■ What body of evidence will satisfy 
inspection requirements? 

■ How do we measure learning (especially 
‘soft’ outcomes)? 

■ How do we compare achievement rates 
across providers whose standards may 
differ? 
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the quality of assessment from the 
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policy formation and strategic planning. 
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PREFACE 



This publication describes the development work from summer 2000 of a 
group of practitioners who have been active in analysing the issues and 
identifying the strategies for recognising student progress in non-accredited 
programmes. The practitioners have sought to communicate the emerging 
issues from their research to policy makers in order to influence their thinking. 
Seven case studies make up the main body of this publication and detail the 
work that has been carried out by these organisations that provide non- 
accredited learning opportunities for adults. The diversity of the provision 
means that readers will need to select from the materials presented in order 
to meet their local needs. 

The centrality of the learners is of key importance and the NIACE research 
on the learners’ perspective is an important contribution to the debate. 

The final chapter identifies a framework for good practice and discusses those 
issues that are common knowledge, the areas in need of further development 
and what further research needs to be carried out in the field of recognising 
and validating adult learning in the non-accredited curriculum. 

Maggie Greenwood 
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1. INTRODUCTION 



1 Policy Contest 

'Whilst qualifications are very important and accreditation will continue to be 
appropriate for the majority of courses, we also want to see the LSC provide 
opportunities with scope for learners to gain recognition for their achievements 
other than through qualifications ’ 1 

1.1 The Learning and Skills Act, 2001 has resulted in the creation of a 
restructured post-sixteen education sector. Within this new sector there 
is a plethora of learning provision for adults, some of which leads to 
qualifications and some for which outcomes are more difficult to quantify 
in a standard form. Whilst recognising the value of non-accredited 
learning and in particular the important contribution that it can make to 
widening participation, 3 the government has made clear its intention to 
raise standards in education. 

The government has now taken steps to promote excellence and to secure radical 
improvements where standards are unacceptably poor. Quality is at the heart of 
our new proposals. The Learning and Skills Council will have a formal role to 
promote high standards and will be expected to fund only learning that meets 
adequate standards . 

1.2 Learning and Skills Councils (LSCs) will have to make decisions as to 
which organisations and learning programmes should receive public 
funds and to what level. The distribution of funds will relate to the 
strategic priorities detailed in the Corporate Plan, March 2001 : 

• To raise achievement of young people aged 19, measured by the 
proportion attaining a level 2 qualification, and the proportion lacking 
the basic skills of literacy and numeracy. 

• To raise achievement of young people aged 21 , measured by the 
proportion attaining a level 3 qualification, and the proportion lacking 
the basic skills of literacy and numeracy. 

• To raise achievement in the entire adult population measured by the 
proportion attaining a level 3 qualification, and the proportion lacking 
the basic skills of literacy and numeracy. 

• To raise participation post-16, measured by the proportion of 16-19 
year olds engaged in education and training. 

• To raise the quality and effectiveness of the education and 
training we support, measured by external inspection grades 
and by structured feedback from learners, employers and 
training providers. 4 
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1 .3 To inform their decisions about the allocation of funds, as well as 
regulate and quality assure education and training provision, a range 
of measures have been developed by the Department for Education 
and Employment {DfEE) and Further Education Funding Council (FEFC) 
in consultation with key organisations involved in the sector. These 
include: a new inspection framework for the Adult Learning Inspectorate 
(ALI) and quality guidelines for the LSC. 

1 .4 In order to secure funding through LSCs it will be necessary for 
providers which include FE colleges, LEAs, the voluntary sector and 
private training providers, to demonstrate that students are making 
educational progress and that robust systems are in place in order to 
ensure continuous quality improvement. There is therefore an urgency 
for policy makers and practitioners to learn from best practice and 
develop a robust system for measuring learning in non-accredited 
provision, which enhances students' experiences and meets the 
requirements of the LSC. The challenge is to develop a system, which 
is sufficiently non-threatening to be valued by people who have had few 
positive experiences of education in the past and/or have minimal 
language and literacy skills, yet be rigorous enough to satisfy national 
standards and the LSC. There are fundamental, practical and 
philosophical difficulties involved in finding solutions. 5 



2 The Project 

2.1 This publication describes the development work of a group of 
practitioners who have been active in analysing the issues and 
identifying strategies for recognising student progress in non-accredited 
programmes and have sought to communicate the emerging issues 

2.2 This is a practical manual for teachers and managers. 6 Examples of 
systems and materials that have been developed by staff in a range of 
adult learning environments are evaluated in order to stimulate lecturers 
and others working with adult learners, to develop systems for 
recognising and validating learning within their own organisations. 

It is also designed to support managers in the development of quality 
systems, which meet the requirements of inspection and funding under 
LSCs. The course and staff development issues that need to be 
addressed are identified and a range of strategies developed by 
leading practitioners are presented within the case studies 

2.3 The first section maps the development of the Learning Outcomes 
Project and progress to date. 
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2.4 The case studies that make up the main body of this publication 
detail the work that has been carried out by a number of different 
organisations who provide non-accredited learning opportunities for 
adults. They are an honest presentation of 'work in progress’ and 
identify tensions and approaches that do not work as well as successful 
initiatives. The diversity of provision within the sector will mean that 
practitioners have to select from the materials presented in order to 
meet their local situation. The evolving models are not intended to be 
prescriptive. However, analysis of the case studies results in the 
identification of some key themes and issues, which require further 
research. 

2.5 The centrality of learners is of key importance; learners are both 
subjects of and participants in identifying achievement in non- 
accredited provision. The NIACE research is investigating the value 
to students of identifying learning outcomes. This chapter looks at 
the issues around involving learners and gives an account of the 
methodology to be used in the investigation. Work with focus groups 
of learners is currently being undertaken and the full research report 
will be published by NIACE in the autumn of 2001. 

2.6 In the final chapter we draw out the lessons from the case studies and 
provide an overview of what we know, what we don't know and what 
remains to be done. The chapter includes a framework for good 
practice, and identifies areas for development, emerging issues and 
questions for further research. 
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2. THE LEARNING OUTCOMES RESEARCH PROJECT 



1 . CONTEXT 

1.1 The Learning Outcomes project which is supported by the National 
Institute for Adult and Continuing Education (NIACE) and the Learning 
and Skills Development Agency (LSDA), developed as a result of 
discussion at the 1999 Further Education Research Network (FERN) 
Conference. 7 Its aims are to: 

« investigate methods of validating achievement in non accredited 
programmes of learning 

• make recommendations on how best to improve the quality of 
assessment from the perspective of funders, educators and students 

• provide evidence on the social and economic benefits of learning 8 

• influence national and local policy formation and strategic planning. 

1 .2 The project has two different but complementary strands. The NIACE 
strand 9 is concentrating on learners as both contributors and subjects 
of systems for assessing learning gains from non-accredited formal 
and informal learning opportunities. The LSDA strand concentrates on 
models of good practice currently being developed by colleges and 
other learning providers. 

1.3 The first phase of the project comprised: an expert colloquium; desk 
research and practitioner consultation workshops which drew on other 
research projects notably the evaluation of the Non-Schedule 2 and 
Adult and Community Learning Fund projects; Raising Achievement 
research at the City Literary Institute and Mary Ward Centre and 
evaluation of the system developed by the Workers' Educational 
Association (WEA). 

1 .4 Through this early work a number of key principles emerged which 
were to inform the development of systems in the case study 
organisations and the formulation of new national policies and 
procedures (see Appendix I). 

1 .5 These principles have been developed into a set of key themes which 
are summarised below and all of which are illustrated in the case studies 
set out in chapter 3. 
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2. 



THEMES WHICH UNDER-PIN BEST PRACTICE 



2.1 The centrality of learners 

• Learners’ needs and expressed preferences should be central 

• Their voices and experience should be evident in the negotiation 
of objectives 

• Students should be involved in the process of assessment. 

• Students should be able to recognise ways in which recording 
learning benefits them. 



2.2 Validation and Assessment, which is ‘Fit for Purpose' 

• Assessment and quality systems should be relevant to 
local situations 

• Not all learning needs to be assessed, although it may be important 
to the learner that it is recognised 

• Systems need to take account of the many forms of personal 
development and the diversity of recording methods required. 

• Validating group, rather than individual, learning may be desirable 



2.3 External standards and clarity about the purpose of 
recording learning 

• Organisations need to be clear about their objectives and differences 
between: assessing learning, validating learning, measuring learning, 
quality assurance. 

• Managers need to decide if different instruments are required 

in order to meet the different recording requirements of learners, 
teachers and funders. 

• Systems should be congruent with FENTO and PAOLO 10 lecturer 
standards and the Student Entitlement. 

• Systems should support institutional self assessment and provide 
evidence for inspectors 
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2.4 Staff Development 

® Management needs to invest adequate resources in the training 
and support of staff 

® Staff need to be involved in the development of local systems for 
recording learning 

• Practice and principles should be congenial to the motivation of 
lecturers. 

» Training needs to focus on identifying a broad range of learning 
objectives, developing a dialogue with students about learning 
and finding imaginative ways to evidence learning 

2.5 The LSDA research to date emphasises the need for managers to 
ensure that the resources required for providing evidence of learning 
are proportionate to the nature of their organisation, the learning 
provision and student profile. The amount of time and support required 
to develop new systems should not be underestimated, especially in 
organisations where there are numerous part-time staff who may not 
have a teaching qualification. 

3. THE CASE STUDIES 

3.1 The case studies, which follow, describe the development of local 
systems for recognising that learning has taken place. In some, but not 
all cases, the learning is also assessed. Organisations were selected 
because they had been involved in the debate concerning achievement 
in non-accredited learning for some time and had begun to evaluate and 
adapt their systems in response to feedback from inspectors and their 
own staff and students. Between them, a range of different adult 
learning environments are represented including Local Authority 
provision, the voluntary sector, FE colleges, external institutions and 
partnerships. Their work spans a broad curriculum range. However, 
developments in ESOL and Basic Skills are being reported as part of 

a separate parallel project and are not a feature of this research 11 . 

3.2 Each case study is presented in a similar format: a context section 
briefly describes the organisation; the learning outcomes model is then 
outlined; the next section describes how the model was developed; 
strengths and weaknesses are then identified and finally each 
organisation presents its conclusions and planned action which will 
develop their model to the next stage. In some cases examples of 
materials used to map learning progress are included in an appendix. 
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NOTES 



1 . Learning and Skills Council Prospectus, 1999. 

2. The FEFC funded some 74 projects over the period September 1999-2000 July 2001 
to deliver innovative programmes which did not require a qualification as an outcome. 
Over 20,000 learners enrolled on programmes in 1999-2000. Ref. Evaluation of NS2 
Pilot Projects 1999-2000 FEDA/NIACE. 

3. Tessa Blackstone, Minister of State for Education in the House of Lords in FEDA, 

Inform issue 3 pp5, 1999. 

4. In the letter of the Secretary of State of 9th November 2000 to Bryan Sanderson, Chair of 
the LSC, recognition was also given in paragraph 26 to non vocational learning - “many 
adults, including large numbers of older and retired learners, will want to pursue high- 
quality and rigorous study for its own sake, and l expect provision to be made to meet 
their needs” 

5. Discussed by A. Hayes, P. Lavender. A.Reisenberger, and J.Vorhaus in Outcomes 
of Adult Learning: taking the debate forward, FEDA with NIACE, 2000. 

6. It is recognised that there are different traditions within organisations concerning the use 
of terms, which describe the people who manage learning these include: teacher, tutor, 
lecturer. Rather than select a uniform descriptor, we have decided to respect local 
practice. 

7. Papers were discussed at the conference, which developed out of research work that had 
been pioneered, by a group of London adult educators who have continued to be central 
to the projects development. They were: Janet Hill, City Lit; John VorhauS and Olga 
Janssen, Mary Ward Centre, Philippa Langton, WEA and Amanda Hayes Kensington 
and Chelsea College. 

8. Research findings from this project are fed into the research into the Wider Benefits 
of Learning being undertaken for the DfEE by Birkbeck College and the Institute of 
Education. 

9. Directed by Cheryl Turner. 

10. PAOLO lecturer standards for Community Education tutors are based on the work 
of Paolo Friere, a radical South American community educator. 

1 1. Recognising and validating learning outcomes and achievement in non-accredited basic 
skills and ESOL. This LSDA project, being undertaken by Sue Grief, parallels the project 
discussed in this paper. It recognises the factors specific to these areas including; the 
national standards for adult literacy and numeracy, the new curricula for adult literacy, 
numeracy and ESOL and the opportunity for LSC funded institutions to claim 
achievement units for learners on non-accredited basic skills and ESOL programmes 
where the primary learning goal is “set wilhin" the national standards. Over 150 
institutions have responded to a questionnaire and five case studies will be 
undertaken. The project report is due to be completed by July 2001 . This will identify 
good practice and make recommendations on ways in which the achievements of 
learners, for whom the new national literacy and numeracy tests are not appropriate, 
can be recognised. 
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APPENDIX 1 



KEY PRINCIPLES 



The principles describe good practice, which should underpin good quality provision for adult 
learners. Managers and teachers can use them as a checklist against practice in their own 
organisations in order to develop local systems to validate and measure the outcomes of 
learning. They may also be used as national standards by those bodies responsible for 
making judgements about the quality of education provided by different organisations. 

The principles support the development of a culture of continuous quality improvement and 
the performance standards already identified in the FENTO lecturer standards and the 
Student Entitlement. 

1. The Learner 

The instrument developed to measure learning is appropriate to the learners’ 
needs and reflects good equal opportunities and inclusive learning practice. 

2. Learning Outcomes 

The personal, social and economic benefits of learning to individuals and communities are 
valued together with planned learning outcomes and are confirmed by lecturers. 

3. Evidence 

Evidence is: (i) collected systematically and adequately robust for external inspection and 
LSC quality evaluation requirements: (ii) available for subsequent accreditation if appropriate; 
(iii) a spectrum incorporating qualitative and quantitative information as appropriate. 

4. Qualify 

The quality framework is responsive to institutional and national agendas. Feedback is used 
to improve quality at course and institutional level and contributes to organisational self- 
assessment. 

5. National Standard 

Practice is underpinned by staff development and linked to FENTO/PAOLO lecturer 
standards. Wherever possible, the system for learning outcomes allows year on year 
comparisons of performance and benchmarking against national performance data. 

6. Bureaucracy 

The bureaucratic underpinning of any system of recording learning and collecting evidence is 
proportionate to the institution's resources and not unduly intrusive of the learning process. 
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3. CASE STUDIES 



THE CITY LITERARY INSTITUTE 

Jan Grant and Wendy Moss 



1. CONTEXT 

1 .1 The City Lit is one of four designated colleges in inner London funded 
by the FEFC to deliver programmes which are, primarily, non-accredited 
adult education. 

1.2 The College operates on three sites in Central London and in other 
community contexts. It offers adult education in: Access and Return 
to Study, Basic Skills, Centre for Deaf People, Humanities, Computing, 
Drama, Health and Fitness, Languages, Music, Psychology and 
Counselling, Section for Independence through Education, Speech 
Therapy, Teacher Training, Widening Participation, Community and 
Parent Education and Visual Arts. An extensive Learning Centre offers 
support for students across all programmes. 

1.3 In 1999-2000 The City Lit offered 2,625 part-time classes and eight 
full-time ones. 5% of these were accredited such as the Access courses 
in Art, Humanities and Drama and a series of certificated courses in 
the Centre for Deaf People, Languages, Music, Health and Fitness, 
Counselling and Computing. 95% of the courses, however, were 
non-accredited. 

1 .4 While courses vary in length between a day workshop to a year or more 
of part-time study, typically a course will range over one term for two 
hours a week. 

1 .5 In the last academic year the college had 22,000 adult students, and 
44,000 enrolments. This reflects the fact that many students will enrol 
for more than one course in any given year. The vast majority of tutors 
are visiting part-time lecturers. There are currently 821 part-time 
teaching staff and 72 full-time or fractional. 



2. A LEARNING OUTCOMES MODEL FOR THE CITY LIT 

Pre-Course 

2.1 Tutors make an initial identification of learning outcomes on the course 
outline sheet. 

2.2 Every effort is made to identify specific learning objectives. It is 
recognised that these will not all reflect the richness of potential 
outcomes, but they are a bottom line for recording achievement. 



2.3 It is stressed that the identification of learning outcomes is one of the 
most important elements of planning and is intimately connected to 
matters of assessment. 

2.4 It is emphasised that learning objectives relate to the learner, and not to 
the teacher. 

2.5 Tutors are supplied with guidance on how to identify specific, learner- 
centred objectives. 

On-Course: First Class 

2.6 Learning outcomes to be presented to learners on a Personal Record 
of Achievement form (Appendix 2). 

2.7 Tutor to discuss learning objectives with learners. Tutor to make a record 
of any amendments to the agreed learning objectives. 

2.8 Learners make a note of their personal hopes on their Personal Record 
of Achievement. They are asked to write down what they hope to 
achieve by the end of the course. 

On-Course: First Class and Beyond 

2.9 Tutors are encouraged to maintain a record keeping system to monitor 
learners’ achievements 

2.10 Tutors are encouraged to think of assessment as an ongoing process, 
but are also advised to set work for learners to which they can provide 
an individual response on at least two occasions during the course. (For 
example: small project; piece of music; quiz; exercise; improvisation, and 
so on.) 

2.11 Tutors make an initial identification of learning outcomes on the course 
outline sheet. Students may see these before the course begins and 
they are given out at the first class. (See appendix 1) 

2.12 At the mid point in the course learners are invited to look over their 
Personal Record of Achievement, so as to consider learning to date and 
to evaluate the course. This may be either an oral or written exercise. 

End of Course 

2.13 Learners are asked to assess their learning, indicating against each of 
the agreed learning objectives how much they have achieved. There 
are four options: 
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I have achieved this; 

I have nearly achieved this; 

® I have started working on this; 
o I can’t do this. 

2.14 Again on their Personal Record of Achievement learners are asked to 
make a note of their personal outcomes. They are invited to answer the 
following questions; 

® How far have you achieved what you hoped? 

® What is the most important thing you have gained? 

2.15 Learners are asked to give examples of any other achievements 
stemming from the course (for example, skills learned, tasks completed, 
personal development). 

2.16 Finally the tutor is asked to provide some written comments, indicating 
on each learner’s Record of Achievement: 

® strengths; 

• points for future development. 

2.17 Learners are also asked to assess the course on a ‘Course Evaluation' 
form (Appendix 3). This includes an opportunity to: 

® rate the course as Excellent; Good; Satisfactory; Poor or Very Poor; 

® provide written comments about the course and/or the teaching; 

® indicate how the learning is to be used after the course; 

<* indicate where, if anywhere, the learner is intending to study next; 

» indicate whether the learner would have liked an opportunity to 
prepare for a qualification in the subject. 

3. BACKGROUND 
Aims 

3.1 Trial new procedures and materials for validating student achievement 
in non-accredited learning programmes at The City Lit. 

3.2 Produce a report and guidelines on the revised framework, which might 
be implemented, after consultation, across the college. 

3.3 Enable consistent recording of retention and achievements on which 
to base clear targets for improvement. 

3.4 Contribute to the national debate on learning outcomes. 
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Participants 

© Ten tutors on a range of courses across the college. 

© One tutor working with students with learning difficulties 

® One tutor working on a speech therapy course. 

© Ten students on each course who undertook to have particular 
involvement and give feedback on the process. 

Strategy 

3.5 To be consultative throughout. 

3.6 Elicit feedback from people occupying a wide variety of roles, including 
senior managers, programme managers, administrators, tutors and 
students. 

3.7 The importance of feedback stemmed from a perceived gap between 
those implementing the system and those experiencing it. This gap 
meant a certain amount of frustration among students, tutors and 
programme managers, which it was important to attend to. 

3.8 It was crucial to learn about the negatives and positives perceived about 
the current system before designing a new one. 

3.9 Feedback seminars were held throughout the project and new forms and 
procedures were designed together with the pilot tutors, with action for 
the feedback incorporated. 

Problems with Past Practice 

3.10 Generally speaking, students had not wanted to fill in forms, had not 
wanted copies of them, and, particularly if they ate into class time, 
resented them. 

3.11 Many tutors gave out forms in the week before the course ended to fill 
in at home. They had little time to complete their part of the form during 
class time during the final pressured classes. The task of photocopying 
forms for students as they left was enormous. 

3.12 The tick box system was confusing. Students found it hard to judge 
where to place themselves. The column ‘I have started to work on this’ 
effectively meant ‘I cannot do this' when interpreted into statistics. 

This did not seem fair to students who might want to see this 

as an achievement. 

3.13 The column* I am working towards this' was interpreted differently 
by different departments. 
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3.14 There were problems with the concept of simple 'achievement' 

in creative subjects: creative process is part of lifelong learning - it was 
impossible to state 'I can do this'. 

3.15 The forms really needed to be done after the class had finished, 
otherwise they did not reflect the full achievement. However, the 
administrative difficulties of sending them out to students after the 
term, was finished was huge. If the students did not see them then 
they were a meaningless exercise as far as they were concerned. 

3.16 The process often did not seem to be meaningful or useful to students 
and needed to relate more to the teaching process. 

Lessons Learned 

3.17 Little use in assessing unless starting points were identified. 

3.18 Need for more focus on what students could do next. 

3.19 The Personal Statement of Learning had to be much more embedded 
throughout the whole course. 

3.20 How the learning outcomes were written was crucial - if they were 
worded too specifically they did not reflect the range of learning students 
achieved. They did not include personal or social outcomes, or unlooked- 
for outcomes (though these were often included in another section). 

3.21 Students in classes for people with learning disabilities often had very 
different individual learning outcomes. 

3.22 The tutor working with students with learning difficulties needed to 
identify individual outcomes for individual students and this was very time 
consuming. 

3.23 One manager commented on the importance of: 

‘separating the value of evaluation to students and the necessity for 
monitoring effort by The City Lit. It is difficult to accomplish both tasks 
without destructive conflict and losing sight of the ultimate aim of The 
City Lit - student learning. If the evaluation process is basically student- 
centred, assessment is too subjective. If evaluation is basically tutor- 
centred, The City Lit drifts towards grading students which can be 
counter productive. On the other hand a self- assessment needs to 
be validated by the tutor to get the full picture of progress'. All this 
emphasised the importance of thorough training for tutors in 
implementing the system. 
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Forms: Suggested Changes 

3.24 Form to reflect a range of outcomes for individual students, including 
personal outcomes (particularly important from an Inclusive Learning 
perspective). 

3.25 The key elements of the form needed to be usable across the Institute 
so comparisons could be made. 

3.26 The form needed to include comments from the tutor, and these to be 
completed. 

3.27 A question for students at the start of the course on what they hoped 
to gain from it and one they filled in at the end to say how far they had 
reached their goals 

3.28 Wider spread of boxes for students to tick, including 'I can't do this' 
so it gave a clearer idea of the range 

3.29 Specific questions on other achievements: including skills, tasks 
accomplished (more concrete than skills) and personal achievements 
so the focus was not only on subject outcomes. 

3.30 Space for tutor comments which was divided into 'strengths’ and 'points 
for future development’ to encourage tutors to be specific in their 
feedback. 

3.31 Description of the purpose of the form, including its funding purposes 
and a thanks at the end (popular with tutors). 



4. REVIEW OF STRENGTHS 

4.1 The whole process of educational discussion and development was 
found to be exciting and much enjoyed by tutors 

4.2 The consultative method employed throughout the pilot, starting with 
“honest” feedback at the level of Senior Heads of Programmes through 
Programme Managers, tutors and students themselves was crucial to the 
pilot’s success. 

We were keen to involve people in as many different roles within the 
college as possible, since informal feedback had suggested that 
previously used Personal Statements of Learning caused frustration 
and resentment amongst part-time tutors, students and, consequently, 
with their immediate line managers who had to implement the system. 
There seemed to be a divide between senior management who had a 
clear idea of what and why the Personal statements of learning were 
necessary, and other staff and students who often saw them as an 
unnecessary bureaucratic exercise imposed from above. We therefore 
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built in a considerable amount of consultation with staff at all levels of 
the college to gather ideas about how we could garner useful evidence 
of student achievement in a way that was clearly useful educationally 
to all involved. 

4.3 The consultation provided valuable criticism and suggestions to build in 
to the pilot. It carried tutors and managers along with the project and the 
educational discussion throughout was exciting, developmental and very 
much enjoyed. 

4.4 Student evaluation of courses always provided important feedback, was 
seen as essential, and was separate from the validation of outcomes. 

4.5 The new question about personal hopes on the learning outcome form 
was, on the whole useful and interesting to students and tutors alike. 
Students felt more involved in the learning outcome process because 
they had thought and written about a personal hope at the beginning of 
the course and they liked reflecting back on this at the end of the course. 

4.6 The focus on learning outcomes at the beginning of the course helped to 
embed the idea of monitoring the achievements of learning outcomes as 
a continuing process rather than an "add on" at completion of the class. 

4.7 The focus on learning outcomes was useful for students and tutors. For 
some students they helped them clarify what they were learning and 
served well to recap at the end of the course. One tutor found that she 
was forced to teach more to outcomes; for example she spent course 
time systematically encouraging all students to try improvising (piano 
class) as she had included this as an outcome. As a consequence, 
students who might otherwise not have bothered because they were 
nervous, tried it out. 

4.8 Students really liked the feedback given by tutors to them individually 
on the forms and in class. They all wanted a copy of this. 

4.9 Whilst in general not liking the forms, and particularly disliking the 
review forms, tutors: 

• like the feedback about their courses 

• think the recording and feedback was worthwhile 

» think the new forms a great improvement on the old. 



5. REVIEW OF ISSUES AND AREAS OF DIFFICULTY 

5.1 Providing comparative statistics on learning outcomes was found 
to be highly problematic. 
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5.2 Some kind of sampling should be considered as tutors, students and 
research organisers were worried about the logistics of the system; i.e. , 
the impossibility of doing very detailed paper accounting on a large 
numbers of short courses. Tutors thought the monitoring and 
assessment of every student was very useful, but were worried 

that it was unrealistic, except in the smaller classes. 

5.3 Learning progress is often easier to detect and measure at beginners' 
level than it is at more advanced levels 

5.4 All tutors had problems with the time taken to complete the procedures 
and the logistics of how they managed to put their comments on the 
forms before the course ended. For example, one tutor felt that her last 
session was crucial to students’ sense of their overall achievement but it 
was also her most pressurised. How could she manage to get the forms 
completed and her own comments included? 

5.5 All tutors and two research co-ordinators found the comparative statistics 
on learning outcomes on the monitoring spreadsheet problematic since 
the results are self-assessed and not comparable, consistent or 
meaningful. 

5.6 We must precede the introduction of new procedures by a training 
programme, as the training has proved crucial in how both tutors and 
students react to the process of validating achievement. There must also 
be a rolling training programme in the writing of learning outcomes. 

5.7 We must be careful in our proposals arising out of the pilot, not to 
introduce procedures that are too bureaucratic or burdensome to tutors 
and students otherwise they will be very hard to implement. 

5.8 We must achieve a consistent system of carefully monitoring our 
procedures on learning outcomes, which is sensible and straightforward 
for those who use it. 

5.9 The monitoring of achievements is always easier for skill based, rather 
than knowledge based courses and one student had particular hostility to 
the use of the procedures in an art history class. 

6. RECOMMENDATIONS AND CONCLUSIONS 

6.1 There will be a number of recommendations from the project, which will 
go forward to the Quality Committee of The City Lit for consultation. The 
improved procedures and materials will be recommended for use in 2001, 
with the changes suggested by the pilot group. It is recommended that: 
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Students 



6.2 Validation of learning outcomes is more embedded in the teaching and 
learning process because the students address their personal hopes at 
the beginning of the course as well as considering the learning outcomes 
as the course goes along. 

6.3 Forms are as simple as possible 

6.4 The students continue to identify learning outcomes achieved. 

6.5 The students state how far personal hopes have been achieved and add 
any gains (expected or unexpected) they would like to mention. 

Tutors 

6.6 The City Lit should, through training and classroom observation, aim to 
improve the quality of assessment on the courses themselves. In 
particular to aim that tutors: 

6.7 Negotiate clear outcomes, differentiated for some learners if necessary. 

6.8 Maintain simple records of students' progress. 

6.9 Ask students to review the learning outcomes during the course and 
make it clear which ones they have been working towards. 

6.10 Include at least two moments on their courses when they are able 
to assess student learning in an appropriate way and give individual 
feedback. This might be through small group discussion and written 
answers to questions, quiz, case study, small presentations, produced 
work, role-play diaries etc. 

6.1 1 Provide quality constructive feedback orally to students throughout the 
course. 

6.12 Provide written evidence that they have given constructive feedback to 
individual students, towards the end of the course either on the forms or 
in their own notes and records. This should list their strengths and areas 
for development. 

6.13 At the end of the programme, to review the evaluation forms and 
complete the tutor review for: 

• what the students had achieved; 

e> what had gone well in terms of content and methods; 

« how problems should be addressed next time. 
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6.14 Additionally the classroom observation might be used for a more on- 
going review with examples of case studies of three individual students, 
their progress, their record keeping and written feedback. This might be 
an alternative to collecting the paperwork for each course. (If too time- 
consuming, it might be part of the course review with tutors in planning 
the curriculum on a rotational and sampling basis.) 

Training 

6.15 All tutors expressed how vital training and discussion were in 
implementing the procedures. 

6.16 A training programme has been planned to introduce the new 
procedures for tutors in 2001, with particular attention to training on 
setting and validating learning outcomes. 

6.17 There is an issue about setting learning outcomes, which must be 
addressed, and also the possibility of setting agreed corporate outcomes 
for particular subject areas. 

Statistical Information 

6.18 The attempt to show comparative numbers of learning outcomes gained 
between courses and programme areas on a monitoring spreadsheet 
should be abandoned as the results can never be satisfactorily rigorous 
or consistent enough to provide a statistically comparable or valid 
overview. 

6.19 Consider whether it is still useful to monitor purely the statistical returns 
on the number of outcomes, evaluations and reviews completed on 
courses over 18 hours as a way of checking procedures are followed. 

6.20 Consider whether multiple and continuing courses should complete 
forms each term or whether a selection or rotation would be an option so 
that students attending multiple courses do not suffer from form fatigue. 

6.21 Consider whether a method of sampling would be an adequate way 
forward or whether it would be too dangerous to abandon the current 
system, and which has taken time and training to achieve. 

6.22 The retention was maintained or improved overall in a year by year 
comparison of the specific courses used on the pilot (overall the retention 
rate in these courses is high). 

6.23 The recording of retention should be added to the spreadsheet. 

6.24 Some way is found of recording enrolments and retention together with 
the information on learning outcomes and evaluation of teaching and 
learning so that it is included in the review process. 
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APPENDSX 1 




PROGRAMME AREA 


HUMANITIES 




PHONE NO 0207 430 0542 


Course title: 


WAYS INTO ANTHROPOLOGY 


Course no: Various 


Course tutor: 


Various 


Day: 


Various 


Dates: Various 


Time of class: 


Various 


Fee: 


Various 


Venue: Stukeley Street 



1 . What is the course about? 

To introduce students to various anthropological approaches in understanding different societies, 
including their own. It will raise some of the current issues in anthropology. 

2. What level is the course? 

The course is for beginners with no experience of Anthropology 

3. What will the course cover? (contents, themes, projects) 

The course is theme-based with a different topic raised each week to give students an idea of the 
range of approaches in anthropology. Different societies will be used to illustrate the themes. The 
choice of themes will be student-orientated to suit their particular interests as far as is possible. 

Religion and Ritual 
Witchcraft 

Kinship and Marriage 

Economic systems: hunter/gatherers, pastoralists 
Development and its relationships to the local peoples involved 
Identity and Ethnicity 

4. What can I expect to achieve? 

- Identify various approaches to understanding different societies 

• Identify various features of different societies 

0 Assess the similarities and differences between custom and belief in different societies 

- Clarify their own ideas and values through participation in group 'discussion 

* Be sensitive to the value of ways of life other than their own 

5. How will I be taught? 

The sessions will run as informal seminars with ample time for discussion and the exchange of 
ideas. Videos, guest speakers and articles by leading anthropologists will be used to illustrate the 
themes. 



^Enrolment Hotline: 0207 831 7831 *Information Hotline 020 7831 9631 ^Advice & Guidance Service 0207 430 0543 




6. What course can S do next? 

Advice on other courses at the City Lit is available during drop-in advice times (see 
prospectus for details ). Appointments for a guidance interview can be made with the City 
Lids Advice and Guidance Service on 0207 430 0543. 

Progression to anthropology degree course in institute of higher education. 

Other related courses in City Lit prospectus. 

7. Arc there any other costs (e.g. certification, materials, equipment or books 1 need to 
buy before or during the course?) 

Students on the daytime course (Terms 1 & 2) can work towards a qualification (London Open 
College Federation). If you are interested please contact Humanities on 0207 430 0542. 

8. How will I know how I am progressing? 

There will be the opportunity for you to write an essay, submit other work of make a presentation 
and receive feedback if you wish. Informal evaluations are also offered. On the daytime course 
(Terms 1 & 2) formal assessment is available through LOCF. 

9. Will I have a chance to give my views on the course? 

At the end of the course (for courses over 18 hours) you will given the opportunity to complete a 
record of your achievements y which will be verified by your by tutor 



^Enrolment Hotline: 0207 831 7831 ‘Information Hotline 020 7831 9631 
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APPENDIX 2 




Persona! Record of Achievement 



This sheet asks you and your tutor to record your learning during your course 
It provides The City Lit with record of our students' achievements which we 
can use as evidence to our funders. It is also a record for you. 

Name: Course Code: 

1. Personal hopes (fill this in at the start of your course) 

What do you hope to achieve by the end of this course? 



2. Learning Outcomes (fill this in at the end of the course) 

These are the learning outcomes you agreed with your tutor at the start of the 
course. Please tick to show how far you fee! you have achieved them. 



Learning Outcomes 


1 have 

achieved 

this 


1 have 
nearly 
achieved 
this 


1 have 
started 
working 
on this 


1 can’t do 
this 































































3. Personal outcomes (see question 1) 

How far have you achieved what you hoped? 

What is the most important thing you have gained? 
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Can you give examples of any other achievements as a result of 
the course? (If you have several others, choose the most significant 
ones for you) 

a. Skills you have learned? 



b. Tasks completed: eg piece of work/performance/projects/book or 
article you have read 



c. Personal (eg confidence, relationships, appreciation of the subject) 



5. Tutors Comments: 
Your strengths: 



Points for future development: 



Sig: (Tutor) Sig: (Student) 

Date: Date: 



Thank you very much for your time 
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APPENDIX 3 



Course Evaluation 



How would you rate the course overall? 

□ Excellent 

□ Good 

□ Satisfactory 

□ Poor 

□ Very Poor 



Please use this space to comment on any aspects of the course and/or the 
teaching 




How will you use the learning gained on the course? 

Further study □ work (paid or voluntary) □ other □ 

If you are going on to further study, are you going to 

□ Study the same course at a higher level at The City Lit 

□ Study another course at The City Lit 

□ Study somewhere else 



If this course did not offer a certificate, would you have liked the opportunity of 
preparing for a qualification in the subject? 

□ Yes □ No □ Not relevant 



Do you have any suggestions for future courses? 




THE CITY LIT 



APPEND8X 4 



FEDA Pilot Study on Validating Student Achievement 
DRAFT 

Course Review and Evaluation Process 
(To be completed by the course tutor) 

(Standard text) 

This report, together with student evaluation forms, is designed to help The 
City Lit identify successes and areas for future development of courses. 

Your programme manager will use this report, together with student 
evaluation forms and registers, to review and plan future programmes. You 
should complete it at the end of the course having read student evaluation 
forms and the record of achievements. Please give it to your Programme 
Head, together with completed student evaluation forms and Records of 
Achievement. 




4. Numbers attending regularly: 



5. Please comment on the attendance and withdrawal from your course. 



6. Evaluation forms 
Please tell us: 

6.1 Number of students completing the form. 

6.2 How many students rated your course 
Excellent 
Good 

Satisfactory 
Poor 



□ 

□ 

□ 

□ 
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Very Poor 



□ 



6.3 How many students are going on to 



Work 



Other 



Further study 



□ 

□ 

□ 



6.4 If going on to study at the City Lit, how many are going to 
Study the same course at a higher level at the City Lit Q 
Study another course at the City Lit [J~j 

Study somewhere else J - ] 

If relevant, how many students would have liked the opportunity to study for a 
qualification? 




What do you think were the strengths of the course? 



What do you think were the weaknesses of the course? 



What will you change next time? 
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7. 



Record of Achievement 



Please tell us: 



7.2 



7.3 



7.4 



7.5 



7.6 



7.7 



The number of students who completed the Record of Achievement, 
nal Outcomes 

How many students felt they had achieved their personal hopes for the 
course? 



How many students ticked they achieved or nearly achieved all of the 
learning outcomes? (Columns 1 & 2) 



How many students ticked they achieved or nearly achieved more 
than half of the learning outcomes? 



How many students ticked they achieved or nearly achieved less than 
half of the learning outcomes 



Please comment on the student response to these questions from your 
perspective. 



Please give some examples of specific achievements students 
mentioned on their forms. 



What can The City Lit do to help you, including any staff development? 







KENSINGTON & CHELSEA COLLEGE PARTNERSHIP 
The Learning Postcard Project 

Amanda Hayes 



1. CONTEXT 

1.1 Kensington and Chelsea College is a Further Education College in 
central London. It is situated in one of the mostly densely populated 
areas in England. Economically, the area presents sharp contrasts 
between poverty and affluence. North Kensington has one of the highest 
rates of unemployment in London. The FEFC has identified the college 
as one of a group, which recruits a high percentage of students from 
disadvantaged areas. 

1 .2 The college primarily serves the needs of adult students (over nineteen 
years old). In 1999-2000 the college had 16,529 students of whom 
7,226 were following non-accredited programmes. Of these students 
80% were female: 20% male (74%:26% in the total college population). 
The age profile was 1% under 18, 80% 19-59 and 19% over sixty. 

13% identified themselves as belonging to minority ethnic groups 
(against 43% in the college as a whole) and 61 1 people indicated that 
they had a disability. 

1 .3 The programme is delivered in four college buildings and a variety 
of venues within the local community. These include local schools, 
community centres, hospitals and clubs. 

1 .4 The college employs more than 700 staff of whom only 20% are 

full time. A high proportion of staff involved in non-accredited learning 
are part-time staff who teach two to six hours per week. Whilst all staff 
are encouraged to attend training events, many staff do not have a 
teaching qualification. 

1 .5 The college budget is approximately £1 0 million with 70% coming from 
the FEFC and 6.7% from the Local Education Authority for non-schedule 
2 provision, as part of the borough's Lifelong Learning programme. 

There is additional income from fees, project work and customised 
training for local businesses. 

1 .6 The college offers programmes designed to meet the needs of all 
groups within the community. The largest curriculum areas in the 
non-accredited provision include: art, craft and design; modern 
foreign languages; cultural and community studies; performing arts, 
health and sport. 
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1 .7 Since 1 993, the education department of the Royal Borough of 

Kensington and Chelsea had contracted with the college and other 
organisations, for the provision of Adult Education. Local inspections 
commissioned by the LEA revealed that the college was further ahead 
in the routine use of quality systems than the majority of community 
groups. Thus, in preparation for funding via the Learning and Skills 
Council, the LEA was keen to work in partnership with the College 
to develop a robust system for measuring student learning outcomes 
in 3 - 30 hour courses. 



2. THE LEARNING OUTCOMES MODEL 

2.1 The Learning Postcard Project was an attempt to identify learning 
outcomes from 3-30 hour, non-accredited adult education classes, 
in a way that would be meaningful to students and lecturers, yet provide 
evidence robust enough to satisfy inspectors and funding bodies. The 
idea behind the project was to make documenting learning as simple, 
non-threatening and familiar as writing a holiday postcard. The card is 
a prompt for discussions between lecturer and students, about learning 
objectives and the learning process, throughout the course. It follows the 
best practice of Inclusive Learning and gives lecturers the opportunity 
to focus on their students’ learning goals and where appropriate, modify 
the content or teaching methods to suit individuals. 



2.2 Process 

2.21 At the start of the course 

• Lecturers explain the purpose of The Learning Postcard 
to their students: 

o Help the lecturer support each person's learning objectives 
o Give students a target and sense of achievement 
o Develop a body of evidence about the quality of learning in 

the college in order to make improvements and ensure continued 
public funding. 

® Lecturers help students identify their learning goals: 

o selected from the course objectives 
o personal targets such as ‘to meet new people' or ‘decorate 
a cake for my parents’ golden wedding’ 
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2.22 During the course 

© Each student’s progress against his or her targets is discussed with 
the lecturer over the duration of the course. 

© Lecturers encourage students to keep other evidence of learning 
such as photographs, audio and videotapes, diaries, samples of 
work. 

® Students are encouraged to reflect on their experiences and identify 
additional ‘incidental’ learning outcomes, which they had not identified 
at the start of the course. 

® Managers make teaching observation visits to monitor the quality 
of teaching and learning and support lecturers in improving their 
practice. 



2.23 At the end of the course 

® The lecturer validates the statements of learning by completing and 
signing the card 

9 The cards are then copied and returned to the students as a personal 
record of their learning. 

» The copies are used by the lecturer as part of the course evaluation 
process and kept as evidence for managers or inspectors 

9 The learning outcomes from the group are summarised (as statistics) 
and areas for course improvement identified in an end of course 
review. 



2.3 The course review for non-accredited courses fits within the overall 
college quality monitoring and improvement system. Each department 
makes an annual self-assessment report to Academic Board. This 
includes a review of performance against targets and an action plan 
for the forthcoming year. 



3. THE DEVELOPMENT OF THE SYSTEM 

3.1 College Quality Assurance (Appendix I - college quality 
framework diagram) 

After incorporation in 1993, the college developed a Quality Assurance 
system which monitored the effectiveness of both FEFC funded 
qualification based work and RBKC adult and community provision. 

It had been a challenge to develop a system which was based on a 
common notion of quality and service as detailed in the college quality 
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policy and student charter, but which embraced the demands at one 
end of the spectrum, of full time externally validated courses and the 
participant ‘fragility’ at the other end, in provision such as family learning 
‘drop in sessions’ or 'first steps' classes in community centres. 

3.2 The system of course quality review used a range of data including: 

• course evaluation by lecturers and students 
® teaching observation reports from managers 
® performance statistics 

® student satisfaction surveys 
® moderator reports (qualifications courses) 

3.3 External scrutiny of learning outcomes in non-accredited 
provision included: 

* Annual RBKC student satisfaction survey 

» Statistical returns on recruitment and retention 
» Inspection 

o RBKC by external consultants 
o FEFC inspectors 

® Moderation by accrediting bodies for FEFC funded courses 

3.4 The annual College Self Assessment Report summarised feedback 
from course reviews, teaching observation and management reports 
and external scrutiny as strengths and weaknesses and identified 
priorities for quality improvement in an action plan. 

3.5 Whilst student achievement was a central feature in the evaluation 
of the quality of accredited provision, this was not always the case in 
the adult and community courses, which did not lead to qualifications. 
RBKC Inspectors reported that the quality system was well embedded 
in the College, but practice was not consistent, especially in relation 
to measuring learning. The biggest challenge was to demonstrate 
‘distance travelled' by students without the introduction of structured 
initial assessment by the lecturer. The inspectors and managers who 
observed these classes identified that in the best courses students 
had grown in confidence, could articulate their learning and had been 
helped to identify ways to take their studies to another level. These 
lecturers understood that skilful discussion of learning is a strong 
motivator in even the most un-confident learner and a major factor in 
good retention. However, discussion with other lecturers revealed that 
some of them saw keeping records of student progress as an additional 
burden and a bureaucratic ‘nonsense’. Some lecturers were vociferous 
in asserting that students did not want records either. It was clear that 
the development of a more rigorous system for recognising and 
recording student achievement would need to gain the full support 

of staff if it was to be successful. 
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3.6 A new system for measuring learning in non-qualification courses 

It was decided that as a first step a minimalist approach would be taken 
which did not require complex documentation. This was especially 
important given the high proportion of second language speakers and 
students with low literacy skills or learning difficulties and disabilities 
(Appendix II - materials adapted with Makaton). The new system was 
however capable of sophistication. For example, the postcard could 
be used for mixed level or mixed ability groups. It allowed students 
to identify their own personal learning goals or select from the course 
objectives provided by the lecturer, it could be used for a one-day course 
or over a longer period. The requirement for the lecturer to validate the 
students’ statements was intended to encourage a dialogue between 
them about learning. 

• Documentation - the learning postcard (Appendix III - 
worked example) 

o wording and layout designed with help of basic skills lecturers, 
o low print-run was arranged so that the cards could be piloted 
with a small group of experienced staff 
o amendments made in the light of feedback from staff 
and students. 

® Range of courses in pilot 

o Subjects - arts and crafts, sport, languages and cultural studies, 
o Venues - college centres, community venues 
o Timing - day and evening 
o Duration - 2 hours per week for 10 or 30 weeks 

» Teachers 1 

o college lecturers including part-time, full-time and curriculum 
leaders 

o RBKC staff 

o staff in voluntary sector organisations. 

3.7 Initial briefing sessions were held at the start of the term followed by 
review sessions at half term and the end of the first term. Lecturers 
discussed the introduction of the system in their classes and identified 
areas of good practice and issues to be resolved. 

3.8 Lecturers were asked to tell students that they were part of a pilot 
project and that the project managers would be interested in their 
feedback on the process 2 . 

3.9 In addition to feedback from lecturers, an internal verification process 
is being developed for more substantial courses where lecturers are 
invited to present case studies of a small sample of students to their 
peers and demonstrate the progress that they have made. 
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STRENGTHS OF THE MODEL 



® The personal, social and economic benefits of learning to individuals 
and groups can be identified and valued together with planned 
course objectives. 

8 Most students in the pilot reported that they found the process 
of setting targets and reviewing learning was motivating. 

© Evidence of performance at student and group level is collected 
systematically 

® In some courses evidence of learning such as art portfolios, 
is available for the students to at a later date as credit towards 
a qualification 

e A spectrum of evidence can be used which incorporates qualitative 
and quantitative data 

6 Fits within a quality framework which is responsive to institutional 
and national agendas 

® Feedback is used to improve quality at course and institutional level 
and contributes to organisational self-assessment. 

® Practice is underpinned by an established staff development 
framework (Investors in People) 

• The system for learning outcomes allows year on year comparisons 
of performance. 

® The bureaucratic underpinning of the system of recording learning 
and collecting evidence is proportionate to the college's resources 
and those of its partner community groups 

• Procedures are not unduly intrusive of the learning process. 



5. WEAKNESSES AND DIFFICULTIES 

5.1 Language ■ There needs to be a shared understanding between staff 
and students about how to communicate ‘learning gain’. 

5.2 Staff development - It has been difficult to find training times, which 
suit part-time lecturers working in the college, RBKC staff and voluntary 
sector partners 3 . 
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5.3 Communicating with students - More needs to be done to 
communicate the positive reasons for measuring learning within all 
courses to students. This will need to be through course documents 
and publicity as well as via lecturers. Lecturers sometimes found it 
difficult to ‘sell’ change in practice to students, some of whom have been 
regular participants of adult learning activities over many years. 

5.4 Evidence of learning - There has been a lack of imagination displayed 
by lecturers as to the form evidence of learning might take. The 
adequacy of the model for external inspection and LSC quality 
evaluation requirements is yet to be tested 

5.5 Audit requirements or administrative Systems/data-base 
information - The College will need to be clear what auditable evidence 
will need to be kept. 

5.6 Benchmarking - Where the collection and comparison of performance 
statistics are required a robust system needs to be developed. The 
capacity to benchmark against national performance data will depend 
on what is required. 

5.7 Course records - Records held by the lecturer, which map individual 
progress against learning goals are of variable quality. There is a 
powerful case to be made for lecturers also presenting case studies e.g. 
two or three per course, which gives a more rounded illustration of 
learning progress - these case studies should include statements made 
by the students themselves in order to reveal the difference that learning 
makes to an individual’s life. 

5.8 Student control of their own learning - Some lecturers tended to 
tell the students what they had learnt rather than engaging them in 
a discussion, which helped them to identify their own progress. 

5.9 Staff attitudes - Lecturers need support if the model is not to revert to 
a ‘token’ paper exercise 

5.10 Timescales - It takes a long time to introduce change, which requires 
a shift in attitude, especially in a climate of uncertainty such as that 
created through post-16 restructuring. 

5.1 1 ‘Distance travelled’ - the model does not include diagnostic testing, 
therefore can not ‘objectively’ identify skills and knowledge acquired as 
a result of attending the course - but then most qualifications courses 
do not do this either e.g. GCSE. 

5.12 Identifying provision that does not ‘fit’ - It should also be recognised 
that some provision of an outreach nature, may not fit happily into any 
framework and to try to force all adult learning into a ’system’ may 
damage some very valuable and effective work. There may be other 
activities, which have a strong therapeutic value to the participants and 
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rather less educational value, such as some exercise classes in centres 
for the elderly. Consideration needs to given as to whether this type 
of provision needs to develop a stronger focus on learning or whether 
it should be funded via a different source such as Health or Social 
Services. Unfortunately the pressure on resources in these services 
means that these classes may not be a priority. It will be important 
to include a number of such courses in the next stage of the research 
in order to identify more precisely what the issues are 4 . 



6. CONCLUSIONS AND NEXT STEPS 

6.1 The centrality of learners 

At its best the model allows lecturers to work with students and help 
them identify and articulate their learning needs and record progress. 
However, some lecturers are not yet developing a dialogue with their 
students and are tending to ‘mark’ them at the end of the course. More 
time needs to be spent talking to students about to find out their views on 
recording and recognising their progress. 

6.2 Validation and assessment 

The model focuses on student achievement but the demonstration 
of distance travelled relies on the quality of evidence collected by the 
individual lecturer. In the best practice, lecturers have found sensitiveways 
to identify students starting points and work with them to record 
increased skills and knowledge. These strategies need to be 
shared with colleagues. 

Concepts about teaching and learning are not necessarily shared, which 
results in mis-communications between members of staff and between 
lecturers and students. 

Some lecturers are assessing their students against national standards 
when this was not the declared aim of the course. Management needs 
to clarify objectives for non-accredited work with staff. 

6.3 External standards 

The model is being modified as information becomes available about the 
way that the Common Inspection Framework will be applied to adult and 
community learning and quality monitored by LSCs. It will be important 
to keep the needs of learners central as concerns about the 
requirements of funders start to impact. 
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6.4 Staff development 



The development of the model has highlighted the amount of time and 
management support needed to successfully introduce new approaches 
to teaching and learning. Administrative staff will also need to contribute 
to system development. 



6.5 Next Steps 

» Review how the system is working with lecturers and students 
involved in the pilot, amend and extend to other staff. 

® Revise college quality system to ensure focus is on learning and the 
impact of teaching rather than satisfaction with the services provided. 

® Get view of external consultant/inspector on the rigor of the process 

» Set up training/dissemination events for other staff 

® Focus teacher training offered to local voluntary organisations 
and college staff on: 

o negotiating the curriculum 
o identifying learning objectives 

o developing creative ways of evidencing learning progress. 

e Develop an internal verification process for more substantial courses 

9 Investigate products available from accreditation boards to evaluate 
their usefulness in assuring quality and benchmarking. 



NOTES 

1 . A high proportion of the lecturers teaching non -accredited programmes are not qualified 
teachers although they may have done stage one training. One of the issues that 
emerged through the project was that a number of lecturers were not sure about what a 
learning objective and were anxious about discussing the learning process. Qualified 
staff experienced in working on accredited programmes also had some problems in 
supporting students identifying their own objectives and incidental learning. 

2. Interviews with groups of students about their reaction to the process are scheduled into 
the next stage of the project. 

3. Stage 1 teacher training in the college now has places available to community 
organisations, with two making use of the offer at the time of writing. Stage 2 will be 
available from Sept 2001. 

4. This provision is currently (Feb 2001 ) the subject of a review by a team of experienced 
managers and an external consultant and inspector using ALI guidelines. The findings 
will inform policy. 
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COURSE PLANNING, QUALITY MONITORING AND REVIEW (Revised September 99) 
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KENSINGTON AND CHELSEA COLLEGE - APPENDIX II 




0 

ERIC 




Welcome to 




Kensington and Chelsea College 



The information on this card will help us to support your learning and recognise your achievements. 



Please fill out this section at the start of your course and give the card to your lecturer. 



This is what I want to learn: VEGETARIAN COOKERY 



1 ) 

2 ) 

3 ) 

4 ) 

5 ) 



Understand nutritional values to ensure a balanced , 
healthy diet. 

Find out about cultural and personal reasons 
for vegetarian diets. 

Develop creative vegetarian meals that my family 
will enjoy 

Develop practical skills to make a range of 
vegetarian dishes correctly. 

Meet new people who like cooking. 



Please complete this section at the end of your course and 
give the card to your lecturer. 



1 can do this 


I'm working 
on this 


1 will work 
on this later 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



This is why I want to learn: 



Other things I have learned/ochieved: 



I have recently moved to the area , and want to 
entertain some new Muslim friends 




Student name: A. Student 



Closs: 



How to get information off the internet. 



Objectives achieved, thinking 
Tutor Comment; about progressing to a LOCN 
course next year. 
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THE MARY WARD CENTRE 



OLGA JANSSEN 



1. THE CHALLENGE 

1 .1 The challenge to us at Mary Ward has been to devise an assessment 
strategy for learning in non-accredited courses which supports the 
learner, encourages reflective practice amongst tutors and satisfies the 
Centre's quality assurance. The strategy must be sensitive to both the 
needs and the views of students and tutors. It should provide learners 
with a constructive and positive sense of their own progress and 
achievements. It should also offer tutors the opportunity to reflect 
on and develop their own practice with the minimum of additional 
paperwork. Finally it should enable the Centre to assure the quality 
of assessment on each of its courses. This paper examines the 
background and production of an assessment strategy that meets 
these disparate goals. 



2. THE MARY WARD CENTRE 

2.1 The Mary Ward Centre is a designated adult education institution, funded 
by the FEFC to provide both schedule 2 and non-schedule 2 courses. 
The Centre has recently increased its accredited provision, but even 
now in the region of 90% of students opt not to pursue certification. 

2.2 The curriculum areas are: an Over 60s' programme; Humanities, which 
includes Health and Physical Exercise, Personal and Professional 
Development, Liberal and Social Studies and Communication Studies; 
English and Other Languages; Computing; and Visual and Performing 
Arts. All 170 tutors are part-time and the total number is equivalent to 
18 full-time teaching staff. The Centre runs over 1,000 courses whose 
average length is 18 hours. 

2.3 The Centre is working to full capacity for its current accommodation, 
with just under 7,000 students and 15,000 course enrolments. Between 
the years 1997 and 2000 there has been a year-on-year growth in 
concessionary fee take-up, which now stands at 39%. During the 
same period there was a year-on-year growth in take-up by non-white 
students, which now stands at 20%. 6.7% students identify themselves 
as having a disability or learning difficulty. The percentage of female 
students has remained at a constant 74%/75% over the last three years. 
Just under 10% of Mary Ward students are under 25. This compares 
with 70% under 24 in the FE sector. The FEFC’s 95/96 postcode 
analysis of students’ addresses showed that 65% of Mary Ward 
students come from disadvantaged areas, which is a very high 
incidence on the national scale. 
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3. ASSESSING LEARNING OUTCOMES: A BRIEF HISTORY 



A Change Of Culture 

3.1 Recently there has been growing pressure from the FEFC inspectorate 
to assess learning in non-accredited courses and we may expect this to 
continue under the new Adult Learning Inspectorate. At the same time, 
the FEFC has stated: ‘...it is not possible to measure outcomes on non- 
schedule 2 courses in any systematic way. ’(Circular 00/01 Annex B 
Paragraph 3) The onus has therefore been on colleges to develop their 
own assessment strategies to suit their local conditions. 

3.2 Formal assessment of learning has not traditionally been required of 
tutors in adult education. Tutors can relate to concepts like ‘encouraging, 
helping and supporting students to learn’ and most are very good in this. 
Indeed the 1998 FEFC inspection of Mary Ward, whilst stating that the 
college had ‘insufficient recording and assessment of students' 
achievements’, graded 79% of lessons observed as grade one or two 
(outstanding or good), compared with 65% nationally. 

3.3 The expectation that tutors of adult education non-accredited classes 
should assess students is a cultural change and, like all cultural 
changes, takes time to be accepted and become common practice. 

It takes even longer in an organisation where all tutors are part-time. 

At Mary Ward, we have experimented with various approaches over 
the last seven years and are still developing our methodology. Many 
of the developments have been conditioned by changes in the external 
environment and we expect this to continue in the current climate 
of change. Our approach has also been dependent on the level of 
resources available to us. Current practice is outlined below and has 
been influenced by the successes and failures of earlier methodologies. 

Why We Abandoned Our Earlier Methodologies 

3.4 We knew from the start that it would not be feasible to keep records 
on all 1 ,000 of our courses, given that the amount of time required to 
measure outcomes would be disproportionate to the very short nature 

of many of the courses. Our requirement was therefore limited to courses 
which were 20 hours or more in length. Initially we used a formal 
approach with standardised written forms to assess each individual 
student's learning. At first these were based on written student self - 
assessment. Later we introduced a requirement for written reports from 
the tutor instead. About a third of the required reports were good. A third 
were less than satisfactory, in that they told us very little about the 
learning achieved. A third of tutors did not submit a report, even though 
we paid them £2 per student assessed. Many of the tutors in the latter 
two categories were in all other respects very good tutors. 
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3.5 Our main problem was that many tutors and students saw the 
requirement as an imposition, to be got out of the way as quickly as 
possible. They could not see its relevance to them; as they saw it: it was 
not in the interests of their teaching and their learning. At best it was a 
task to be tolerated in order to secure funding. Many students as well as 
tutors had come into non-accredited adult education precisely because 
of the freedom from formal assessment. Had they wanted assessment, 
they would have taken courses leading to qualifications. 

3.6 As education managers we are convinced that assessment of learning is 
an essential component of successful teaching and learning. But try as 
we might to introduce a mandatory requirement for written records for all 
students across the curriculum, painful experience had taught us this 
was not feasible. For students and tutors it was not even desirable. If we 
were to win over tutors and students, an informal system with a minimum 
of paperwork would be necessary. 



Finding Out What Students Wanted 

3.7 In 2000, as part of its inclusive learning initiative, the Centre decided to 
research students’ perceptions of their needs for assessment. Then we 
could make our assessment match more closely what students wanted. 

A series of student focus groups were undertaken to investigate this. 1 

3.8 What the students told us was not only informative: it challenged some of 
our preconceptions. The key points for students were: 

• Getting honest, detailed and regular feedback from their tutor on their 
progress and advice on the next step. They also wanted more of it 
than they were getting from their tutors. 

. Very few students wanted any formal or written assessment. 

. Frequent informal, formative assessment was their priority. 

Summative assessment mattered less for students who were not 
learning for a qualification. 

. Self-assessment had limited value for them; it became meaningful 
only when combined with tutor assessment. 

• Students were happy to talk at length about their learning, but they 
did not want to fill in forms about it, unless this was a requirement 
of funding. 

3.9 Our students were giving us a steer to move away from concerns 
about a written record of what they had achieved. They were advocating 
an approach which would emphasise informal, continuous, on-course 
assessment of progress. They also wanted us to prioritise active 
interchange between tutor and students on how they were getting 

on and what to do next. 
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4. ASSESSING LEARNING OUTCOMES: THE MARY WARD MODEL 



The Model 



1 . Write a course description, which the Head of Department approves, 
and a copy of which is given to all students on the course. 

2. Make an initial assessment of student needs. 

3. Make on-course assessment of student progress. 

4. Make a final assessment of student achievements and advise 
students on the next step 

5. Write a written report on each course, using a simple standardised 
form. 

6. Write three examples of assessments of student learning in any one 
of their courses, to demonstrate the range of learning on that course. 

7. Present these assessments at a meeting of tutors for peer 
moderation and verification by a Head of Department or another 
experienced assessor. 



How The Model Applies To Our Tutors 

» All tutors, irrespective of course length, required to complete stages 
1-4. 

• Stages 2 to 4 must be shared with students, but a formal written 
record to the student is not required. 

• In addition to stages 1-4, tutors teaching courses of 20 hours or 
longer are required to complete stage 5. 

• In addition to stages 1-5, all tutors engaged for 20 hours or more per 
annum, irrespective of the length of their courses, are expected to 
complete stages 6 and 7. 



5. THE MARY WARD MODEL: STAGE BY STAGE 
Course Description - Stage 1 2 

5.01 We already had a very well established house requirement that all tutors 
write a course outline which is available to all enquirers from the moment 
the annual prospectus of courses is published. This includes a section on 
learning objectives for the course: ‘what you should be able to do by the 
end of the course’. The primary purpose of this document is to provide 
guidelines for students about what they can expect when they enrol for 
a course. At the first meeting, all students receive a copy and the tutor 
is expected to go through the course outline. There is an opportunity to 
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amend the objectives, both at this stage and subsequently in the light 
of experience as the course progresses. Since non-accredited courses 
do not have a prescribed syllabus, the principle of negotiation of the 
content between tutor and students is an important one. What matters 
is that the learning objectives are clear and are agreed by the students. 

Initial Assessment Of Student’s Needs - Stage 2 

5.02 Tutors are expected to make an initial assessment of their students’ 
needs. This can be a whole class, public activity or a private student- 
tutor dialogue, as appropriate to the subject, the sensitivity of the 
information and the time available. The assessment involves finding out: 

» Why students have chosen to do the course; 

» What previous learning, skills, knowledge, including qualifications, 
they have in this and related subjects; 

® What their future aspirations are once they have completed 
the course; 

» What learning difficulties or disabilities they may have and 
what special support, if any, they require. 

5.03 Whilst we would want tutors to keep notes on this initial assessment, and 
many of them do, there is no mandatory requirement for them to do so. 
The required evidence of the assessment is limited to the three individual 
student assessments, as explained below. 

On-Course Assessment Of Student Progress - Stage 3 

5.04 Throughout the course we expect tutors to assess students’ progress 
against the agreed learning objectives. Guidelines are given to them on 
a range of informal and formal strategies, some paper based, some not. 
Tutors may choose existing strategies appropriate to their course and 
students, or develop their own. Our experience has been that some 
tutors are very imaginative in devising assessment strategies and to be 
prescriptive would inhibit flexibility and creativity, two of the real strengths 
of uncertificated teaching and learning. Others are only too glad to have 
a menu of options to choose from. These will form the basis of their 
on-going feedback to and discussion with students on their progress. 

As with the initial assessment, documented evidence of on-course 
assessment for all students is not a requirement, other than in the 
three student examples. 

End-Of-Course Assessment And Advice On The Next Step - Stage 4 

5.05 At the end of the course, we ask tutors to discuss with students what 
they have got out of the course. Very often students will speak of 
additional benefits, not related directly to the course learning objectives. 
These may include things like personal skills, learning skills, social or 
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economic benefits derived from the course. At the same time the tutor 
will confirm what they have achieved relative to the course’s learning 
objectives. Finally the tutor will discuss with students what their next 
steps might be. These will vary according to the student’s aspirations. 
Experience at Mary Ward is that few students in non-accredited 
programmes are looking for entry to further or higher education or for 
career advancement. The majority will be looking into the options for 
more non-accredited learning. As with the initial assessment and the 
on-course assessment, the time spent on this final assessment may 
be in plenary session or one-to-one with students, as appropriate 
to the subject and the length of the course. The only documented 
evidence required of this assessment is in the three written-up 
examples of student learning. 

Written Course Report - Stage 5 3 

5.06 Where a course is 20 hours or longer, within one week of the course 
completion, tutors are asked to submit a written report on their course, 
using a simple standard pro forma. This has three sections: student 
retention; student achievement; course evaluation. 

5.07 The Centre holds statistics on student retention for all its courses. The 
reason for requiring in addition a commentary from tutors is two-fold. 
Firstly, it is to establish with tutors that retention of students is important 
and a measure of the success of the course. Secondly, our statistics 
do not tell the whole story and we want to safeguard against any false 
interpretation where retention is poor. Tutors are asked to say why 
students have discontinued. The Centre support staff operate a 
telephone follow-up service of absent students and a good tutor will 
usually know why students are no longer coming. Often the reasons 
are beyond the tutor’s or the college’s control. 

5.08 For achievement, we ask tutors to indicate against the register of 
attendance for each and every student what their achievement is on 
a four-point scale, as described below. No supporting documentary 
evidence needs to be submitted. 

5.09 In section 3 of the course report, to assist curriculum planning, we ask 
the tutor to complete an evaluation of 7 key aspects of the course, with 
supporting comments. 

5.10 The course reports are just one of the devices used to inform the 
Centre’s course review. Other sources of information are: departmental 
and course retention rates; class visits made by the Head of Department 
to assess the quality of teaching and learning; feedback from students 
about the quality of the course through student satisfaction surveys as 
well as through informal conversations. We also have a survey of 
students who have discontinued, based on telephone interviews. All 
these sources of evidence are collectively used not just for course review 
but also for tutor appraisals and in-service training. The Mary Ward 
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strategy for assessment of learning operates alongside all these other 
quality assurance measures. 

5.11 Like the course description, the course report is now a contractual 
requirement, for which tutors receive no additional pay. We are in our 
third year of course reports and although we have not yet quite achieved 
100% completion, both the number of returns and the quality of 
responses have improved since our early days. 

A four-point scale of achievement 

5.12 We have two measurements: progress made by the student against 
learning objectives and the ‘added values’ for students from attending 
the course. These two aspects combined show the 'distance travelled’ by 
the student. 

5.1 3 Our four-point scale is based on a qualitative assessment of the 
student’s learning. At the same time, since it is a graded scale, we do 
have a quantitative measure of the learning that students derive as a 
whole from the course. The four point scale is: 

1 . The student has made substantial progress against all the learning 
objectives and may have gained additional benefits from attending 
this course. 

2. The student has made significant progress against some of the 
learning objectives and may have gained additional benefits from 
attending this course. 

3. The student has made some progress against the learning objectives 
and/or has gained additional benefits from attending this course. 

4. The student has not progressed against any of the learning objectives 
and has gained no additional benefits. 

An example of the four-point scale in practice 

5.14 Critical to the understanding of the 'distance travelled’ concept is the 
way in which non-accredited adult learning differs from accredited 
programmes. Most courses have no entry requirements and students’ 
starting points vary enormously. Secondly, when students are not 
learning for a qualification, the additional benefits are often more 
important to them than achieving all the course objectives. Thirdly, for 
many students on these programmes learning is a life-long commitment. 

5.15 Unlike accredited courses, the measure of achievement is not the level 
of attainment achieved by the course end, but the progress made by the 
individual student relative to that student’s starting point : the ‘distance 
travelled’. And, unlike accredited courses, the additional benefits or 
'added values’ are also recognised as achievements. 
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5.16 This can be illustrated by two students attending the same life 
drawing class: 

One is a graduate in fine art who wants to refresh her life drawing skills, 
the other is a complete novice wanting a taste of what life drawing is all 
about. The former is reactivating skills acquired some time in the past; 
the latter is learning for the first time about proportion, perspective, use 
of materials, mark making etc. Relative to their starting points, the novice 
might progress against all the course’s learning objectives, though the 
level of execution in the completed work of the graduate is much higher, 
and so our novice merits a grade 1 in our definition. Yet the graduate 
might have been suffering from a creative block, which was released by 
the life drawing course, enabling this student, while technically no more 
competent at life drawing than at the start, to make a real break-through 
in confidence, an undoubted 'added value’ for that student. And if the 
learning objectives included alongside the technical competences: 
demonstrate a willingness to take risks; explore and experiment with 
different approaches; engage in critical evaluation of their work; and all 
of these were gained by the graduate on the course, then we might 
place her as grade 2. 

5.17 Students are not informed of their ‘grade’, but are informed of the 
underpinning assessment. This is because what matters to them 
is the quality of the feedback they receive on their progress, not a 
quantitative measurement of it. Our grading is a quality assurance 
exercise, to ensure a consistency of understanding of what is meant 
by ’achievement’ and a yardstick of the numbers achieving each of 
the four grades. 

Documented Examples Of Assessing Learning - Stage 6 4 

5.18 As evidence of the tutor’s assessment practice, each tutor is asked 
to write up three detailed examples of student learning, from any one 
of their course that year. They should illustrate the range of learning 
within the class. 

5.19 We use a standard pro forma for the examples. This covers: 

9 The assessment strategy used by the tutor for the initial assessment, 
on-course assessment, end of course assessment as well as advice 
on the next steps. 

» The initial assessment made by the tutor on the student in question. 

® An assessment on the progress made by the student against the 
course objectives. 

9 A description of any ‘added values' or additional benefits derived 
from the course by the student. 
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® Advice on the next step for the student. 

o A placement of the student in one of the Mary Ward four grades of 
achievement. 

5.20 Tutors are given two hours’ pay for the drafting of these three written 
assessments. In our pilot of this scheme, we found that tutors took 
between two and four hours to write the assessments, but became faster 
once they had got the hang of what was required. The quality of these 
assessments was in nearly all cases very high. Tutors presented their 
assessments to fellow tutors at peer moderation meetings, described 
below. Our Inclusive Learning Facilitator also attended and commented 
later : ' I thought the tutors handled the whole issue as a formative 
process i.e. part of learning and not part of college paperwork and that 
this was excellent and should be emphasised in any written guidelines. I 
also thought they demonstrated considerable care and empathy with 
their students, which again was wholly professional and is the meat of 
what we are talking about.’ 



Participation In Peer Moderation Meeting - Stage 7 5 

5.21 Tutors are expected to attend a tutor peer moderation meeting, chaired 
by a head of department or an experienced tutor assessor, who has the 
role of internal verifier. Levels of staffing at the Centre do not permit the 
roles of chair and verifier to be split between two people as we would 
like. At the moderation meeting, the tutors read the three student 
assessments and then have an opportunity to ask questions about the 
assessments in order to make a judgement about the quality of 
assessment using the moderator's form to record their evaluation. 

5.22 The purposes of this peer moderation are: to share and disseminate 
good practice; to problem solve in difficult assessment cases; to achieve 
a consistency of approach and judgement. To date we have conducted 
three meetings, as a way of piloting the scheme, with a total of 18 tutors. 
Although we are still developing the format of the moderation meetings, 
we are confident that the principle of tutor peer moderation is a good one 
because it encourages tutor reflection on good teaching and assessment 
practice and is an opportunity for continuous professional development 
of our tutors. 

The Group Moderation Process 

5.23 The group moderation process is as follows: 

» The meeting consists of a chair/verifier (either a Head of Department 
or another approved verifier) and 4/5 tutors. 

® The chair outlines the process of the meeting to those present 
and the moderation process. 
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® The chair is clear about the timing of the meeting: 2 hours split into 
10 minutes introduction, 20 minutes for each tutor’s presentation and 
moderation, 10 minutes to sum up. 

» Each tutor distributes copy of their three documented student 
assessments to the group for reading and discussion, answering 
questions prompted by the moderation criteria. 

® Immediately after each discussion the verifier, and the 3/4 other 
tutors, complete a moderation pro forma. 

» After the meeting the verifier grades the tutor's assessment, making 
use of the moderation pro formas and confirms or adjusts the grade 
awarded to the students according to criteria agreed by the Centre. 

5.24 Tutors are given feedback by the verifier on their assessments and 

advised what, if anything, needs further development in their assessment 
practice. Tutors are paid the standard hourly tutor rate for the amount 
of time spent at the moderation meeting. Those tutors who achieve 
a moderation report which gives them a good or excellent profile are 
considered to be 'approved' assessors. We then take on trust the 
judgements they make about student achievements, just using the 
grades and without documentary evidence on all students, Tutors 
who do not achieve approved status are given support to improve 
their practice. 

6. INSPECTION FRAMEWORK 6 

6.1 The Mary Ward strategy for assessing learning has been matched 
against the quality criteria of the ALI and Ofsted Common Inspection 
Framework for inspecting post-16 education. How the evidence 
generated at Mary Ward meets the criteria is explained in the grid in 
Appendix 5. 

7 CELEBRATING ACHIEVEMENT 

7.1 One of the most fulfilling aspects of non-accredited learning for students, 
tutors and providers is its power to promote creativity. Alongside our 
assessment of student learning, we feel it is important to take every 
opportunity to show-case and celebrate student achievements in 
imaginative ways. Displays of student work in the Centre, public 
performances and presentations - both in-house and outside in our 
local community - publications of student writing, a 'student successes' 
notice board, sound and video recordings of student work - all these 
contribute to a sense of pride in what students have achieved. 

7.2 The NIACE individual student and group awards have done much to 
raise the profile of what adult students achieve. With no 'certificates’ 
to show for their learning, just how much students do achieve is all too 
easily hidden. At Mary Ward we have started to address this lack of 
acknowledgement by introducing our own Mary Ward awards, based on 



the NIACE individual and group models. Nominated students have 
improved their own lives and/or the lives of others as a result of their 
learning experience. It is not the level of learning that is important, but an 
awareness of the benefits the learning has brought and the evidence of 
personal achievement. These Mary Ward certificates are awarded at a 
public ceremony attended by students, their family and friends, as well 
as staff and governors. Our experience has been that those who really 
appreciate this public acknowledgement are those who have struggled 
most to achieve and in many cases have never been awarded a 
certificate before. 



8. EVALUATION 
Strengths and Benefits 

» Tutors encouraged to identify course aims and objectives, and 
reflect upon and improve their role as assessors of learning 

® Assessment and reflection upon learning incorporated into 
process of teaching 

» Better understanding of student needs for assessment in 
uncertificated courses 

e Moderating samples of assessment is a good opportunity for staff 
development, and provides an opportunity for tutors to learn from 
their peers 

» Opportunities for questioning practice, and discussion of examples, 
gives a more rounded picture than written documentation on its own, 
and puts those tutors who find written documentation difficult at less 
of a disadvantage 

• The moderation and verification process means that assessments 
are validated 

• Some models of good practice in assessment of learning 

• Higher profile of ‘student achievement’ at the Centre 

• On-going commitment to giving students quality feedback on 
their progress 

» The assessment of learning contributes to course review 

• Benchmark for achievement in the Centre’s uncertificated courses 

® Assessment process can be made congruent with quality assurance 
procedures 

® The strategy matches the quality criteria of the ALI and Ofsted 
inspection framework. 
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Weaknesses 



e Some tutors and learners find it difficult to focus on learning, as 
distinct from, e.g., course evaluation. 

© Paperwork/repetitive ‘tick-box’ tasks can (i) intrude into teaching and 
learning; (ii) lead to diminishing returns (owing to ennui, etc.) 

® Some judgements difficult to assess - e.g., levels of confidence and 
self esteem 

© Student self-assessment sometimes left to stand without tutor 

comment; tutor assessment sometimes passed to management only 
(ie.not also to students) 

© Some tutors struggle with assessment, either because they are bad 
at assessment, or because, though good at assessment, they 
struggle to present assessment in the form of a written report 

© Organising peer moderation where all staff are part-time is difficult 
and time consuming 

Issues 

© A remaining danger that tutor assessment is thought of as a discrete, 
end-of-course activity, as opposed to an activity that is integral to the 
teaching process 

» The relative priority given to assessment of learning, and the auditing 
of that assessment, as distinct from other pedagogic tasks, e.g. 
instruction and practice 

© Stakeholders' interests are polarised. On the one hand, students do 
not seek documented, formal, summative assessment, but do want 
informal, qualitative assessment. On the other hand, funders may 
well want a formal, summative assessment or at least some 
quantitative indication of the levels of student ‘achievement’ in 
uncertificated learning 

© Devising a reliable yard-stick of learning within the same class, given 
the range of variables to be taken account of (for example, a locally 
negotiated curriculum; multi-level learning) 
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9. THE PAY-OFF FROM A SYSTEM BASED ON EXPERIENCE 

9.1 After seven years developing a system, we are confident that we have 
now arrived at a strategy which is both pragmatic and feasible for the 
Mary Ward ratio of students to staff. There is no mandatory requirement 
either for tutors or for the college to hold documentation on the 
achievements of all students. Many tutors will hold their own 
documentary evidence, but we recognise that there will be a minority of 
otherwise excellent tutors who do not. The trade-off for the abandonment 
of a mandatory requirement for written records is the quality feedback to 
students on their progress, and the moderation and verification of 
standards of assessment. The Centre will continue to refine and fine- 
tune its practice. We are now much closer to a system which meets the 
needs of all the stakeholders: students; tutors; managers and, hopefully, 
funders too. 



NOTES 

1 . Findings summarised in Janssen, O., 'The Students' Voice', 'Adults Learning', NIACE, 
October 2000 Volume 12 No 2. 

2. See Appendix 1. 

3. See Appendix 2. 

4. See Appendix 3. 

5. See Appendix 4. 

6. See Appendix 5. 
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GO 



The Mary Ward Centre 

- the friendly place to learn - 



Appendix 1 



Course Description, Session 2000-2001 - 559 SHIATSU FOR BEGINNERS 24hrs 

Vcw course/ tutor will di^tuAythiy with yoa at the, beginning, of the, cover se,. 

Who this course is for and what previous knowfledge/experiencc/skiMs we expect students to have 

The, course/ (a open to anybody, vneuy find, it difficult to 

work, on the/ floor. }C op reAioxey experience, or knowledge, iy required/, only a,wMingy lesyto 
learn. For studenty who have, Fnglish oa cv s<uuond,laA^gujx^ of spoken English, 

and/ good/ cowcprehension are, a/ prerequisite/. 



Course content 



Shiatyu/ meuny 'finger pressure/. It iy a, bxoxdy therapy, bt Med/ on the, s<ume,principleA oa 
acAOpuruXure,, fiyxoLAing, on the, internal energy of'KO’. Vo-In (self-Shiutyu) ewcto(m<i 
\neditation ares included/ in the , coursey to- increase, fitneAy, self- aw arenexS', perception / of 
energy and/ the, ability to give, Shiatsu,. 

The, course/ cUAnyto give, a baxio under standiA^ of Shiatsu,, and, w 0d teach, you, how to 
give,a/full/boxiy Shiatsu/ working. On prone,, sup Ore, and sitting, poytTuy^ 



What materials you will need for the course and estimated cost 



Blanket, cuAhion, notepad/ and/ pen/. 

Loose*, comfortable, clothings should, be, worn. 



What you should be able to do by the end of the course 



Z. Work, in, the/p rone, position, (on the, bach). 

2. Work, in, the, supine, position (on the/ front). 

3. Work, in, bitting position. 

4. Qbe, a, siuyxple/fuU, lx>dy Shiatsu treatment to family anbfriendA- 



The teaching and learning methods we use on the course 

The > course/ iA taught through, demonstration asnds p r(Act0ccb application with the, 
Studenty workings on each, other. 



Ho>v we assess your progress on the course 

They tutor obsenveA studenty 7 practice, and/ offers' ad*/ Lee, while/ they are, doing' the, 
practlcaly- Ue,alsogiieAtutoriaLfeedlyachona,one,-to-one,baAOy. Students- are, 
encouraged/ to contiruucMy evaluate, their own posture, and/ comfort while, working,. 



What extra study/practicc wc expect you to do outside the class 



N one, Oy expected, btAtprcActice/ of the, technigueAcnfar^ and/friendx wilt help progress*. 



What you can do next, after this course 



Further Shixvtsu<xnAr sey at bl ary Ward or professional Shixctsaschooly. The, tutor will/ 
advise,. The, Centre, also runs* introduction to (Acupuncture, and, a, range, of courseA On 
alternative medicine,, Ixrth, accredited, and, non- accredited/. 
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Mary Ward Centre 



Appendix 2 



Tutor’s Course Report 

Deadline tor receipt by Head of Department 21/12/00 

Course code and Title ... 559 Shlat^for Beg^wiery. Academic Year.. .00/01... . 

Course Tutor’s Name T T Date of report... 15/12/00 

Duration of the course in hours 24 

Please note: it is a contractual requirement that you complete and return this form to your Head of 
Department within one week of the course ending. 



Section A Student Retention 



The Centre database will give us a retention rate for your course. You, as the course tutor, may know 
why students have withdrawn or not completed the course. Please give us below information or 
observations you have on reasons for students discontinuing. Indicate too the number of students who 
discontinued for any reason. For instance: 2 students - work commitments. 1 student - course 
unsuitable. 



6 - 

3 duuc ter work/ coin^nltwicvty They KcuL C^utuMy been/ ahlctolecvve/ 
worh early for thiy4 o clocks elewy, but iri/prcMttcc found/ it very 
difficult. 

1 dj^xUxiextShioct}^ WCL& not for her. 

2 not hno\vv\/. 



Section B Learning Outcomes for Students 



1 . Please make a photocopy of the class attendance register towards the end of your course or at the 
final meeting Indicate alongside each name one of the following: X; 1 ; 2; 3; 4. In the boxes below left 
give the number of students awarded that grade’ and total at the bottom. 



Note: where a student has attended this course before, it is important that you make an assessment on 
progress made within the timespan of this particular term or year’s course, as appropriate. 

2. Please attach to the register a copy of the course description. 



r X : No assessment, the student's attendance or participation was insufficient for you to make an 
assessment 



4 

2 

0 

17 



1 ; The student has made substantial progress against aH the learning objectives and may have gained 

additional benefits from attending this course 

2 : The student has made significant progress against some of the learning objectives and may have 

gained additional benefits from attending this course 

3 : The student has made some progress against the learning objectives and/or has gained additional 

benefits from attending this course 

4 : The student has not progressed against any of the learning objectives and has gained no additional 

benefits 

Total 
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App endix 2 



Course Code: 559 Title: SHIATSU BEGINNERS 

Tauter: T. T. 

Day: Thursday Times: 18.00 to 18.00 Total Hours: 24 

Start Date: 21/09/00 End Date: D4/12/0Q Number of Meetings: 12 



Class Maximum: 18 Room Number: 33 Term: 1 



Student 

No 




Date 

Achievement 

Grade 


21/ 

9 


28/ 

9 


5/ 

10 


12 1 
10 


19/ 

10 


26/ 1 
10 ! 


9/ 

11 


16/ 

11 


23/ 

11 


30/ 

11 


7/ ! 
12 


14/ 

12 


Name 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 




12 


1 7878 


Student 1 


X 


/ 


o 


o 


A 


w 
















2 7979 


Student 2 


1 


/ 


7 


( 


/ 


/ 


f 


o 


/ 


o 


(' 


/ 


/ 


3 8080 


Student 3 


1 


/ 


7 


7 


/ 


/ 


( 


( 


/ 


E 


f 


o 


r 


4 8181 


Student 4 


X 


/ 


/ 


/ 


f 


/ 


[ 


w 










5 8282 


Student 5 


1 


/ 


{ 


/ 


E 


/ 


i f 


/ 


/ 


/ 


f 


/ 


/ 


6 8383 


Student 6 


i 

2 


o 


o 


/ 


/ 


/ 


E 


/ 


7 


( 


f 


/ 


( 


7 8484 


Student 7 


X 


f 


f 


f 


/ 


/ 


W 














8 8585 


Student 8 


1 


... { 


{ 


f 


/ 


/ 


f 


o 


o 


{ 


f 


/ 


( 


9 8686 


Student 9 


X 


o 


f 


' w 




















10 8787 


Student 10 


1 


( 


( 


. f 


/ 


/ 


{ 


/ 


o 


{ 


f 


/ 


f 


11 8888 


Student 1 1 


3 


o 


o 


o 


A 


E 


E 


/ 


f 


{ 


f 


/ 


7 


12 8989 


Student 12 


2 


{ 


{ 


E 


E 


/ 


/ 


/' 


f 


O 


o 


/ 


{ 


13 9090 


Student 13 


2 


{ 


f 


/ 


/ 


/ 


E 


f 


f 


/ 


( 


E 


o 


14 9191 


Student 14 


3 


o 


f 


f 


E 


1 / 


O 


7 


r 


E 


o 


/ 


/ 


15 9292 


Student 15 


X 


7 


f 


7 


W 














16 9393 


Student 16 


X 


{ 


{ 


w 
















i 


17 9494 


Student 1 7 


2 

1 


o 


/ 


( 


O 


/ 


o 


f 


o 


/ 


f 


/ 


/ 


Total Number Attending 




12 


: 14 


12 


: 5 


12 


1 7 ■' 


9 


8 


7 


9 


9 


1° : 


Tutor Initials 




i rr 


ITT 


rr 


1 tt 


rr 


rr 


TT 


rr 


rr 


• rr 


rr 


TT 


/ = present; O = absent; V = visitor; E = 


student excused; W = student withdrawn; A : 


= absence enquiry requested 



Achievement Grade: 

X : No assessment, the student's attendance or participation was insufficient for you to make an assessment 

1 : The student has made substantial progress against aN the learning objectives and may have gained additional 

benefits from attending this course. 

2 : The student has made significant progress against some of the learning objectives and may have gained 

additional benefits from attending this course. 

3 : The student has made some progress against the learning objectives and/or has gained additional benefits 

from attending this course. 

4 : The student has not progressed against any of the learning objectives and has gained no additional benefits. 
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Section C Tutor's Evaluation of the Course 



Assess by circling the appropriate number on a scale of: 

1 : entirely true; 2 : mostly true; 3 : partly true; 4 : not true 



Please write comments alongside to support your score, where appropriate. 



Statement 



Score 



1 . The objectives given on the course 

description were appropriate for the class. 



12 34 



Comment 



2. I was able to cover all the contents given 
in the course description. 



1234 



3. Students were appropriately placed on 
this course. 




4. The room allocated, equipment and 
materials for the course were appropriate. 






5. The course length and timing were 
appropriate. 




6. The teaching strategies I used were 
appropriate. 



12 3 4 



7. There is nothing I would change on this 
course if I were to teach it again. 



1 2 3 4 



3. One/ itudert d&clded/ 
Shiatyu/ way not for her. 
One/ yCicdent way 
dt^abled/£r anable' to 
Shlatkusy but I 
dernontfr(&ted/ on 
htm/Er he/ learned/ a/ hot 
about the/ therapy. 



4. Room/ 33 way adequate/, 
but the/ carpet way not 
alwayy ay clean ay J 
would/ Kaye/ for floor 
eyerctt&y. 

5. TirnCn^ proved/ cliffy 
On practice/ for thoyc/ 
corning from work/ 

(4pm/ ytart). Cow&length/ 
appropriate/. 



7. I like/ to- introduce/ 
yornethings new Lneyery 
course/. But in principle/ 

I* m/happy with/ the/ coterie/. 
The/ practical/ ernphayly 
iy appropriate*. 



Section D Optional Additional Comments 



Add, if you wish, any information which you think the department or Centre should know about how the 
course went. This will help your Head of Department support you and plan for the future. Mention 
anything which should be taken into account when reading sections A to C above. 

Within a week of your course ending, please return the course report to your Head of 
Department. 



Mary Ward Centre 

Assessing Learning - Student Case Study 



Appendix 3 



Course Code and Title: 5 59 Shiatyj/ 

Course Tutor’s Name:,.T.r 

Date of Assessment:. 1 5/1 If 00 

Outcome for this student: 1 



Start Date: 2 2/9/00 End Date: 14/12/00 

Duration of Course (hours): 24 

Student Example Number:..,. 2 




1 : The student has made substantial progress against aj] the learning objectives 

and may have gained additional benefits from attending this course, 

2 : The student has made significant progress against some of the learning 

objectives and may have gained additional benefits from attending this course. 

3 : The student has made some progress against the learning objectives and /or has 

gained additional benefits from attending this course. 

4 : The student has not progressed against any of the learning objectives and has 

gained no additional benefits, 

1. Description of Assessment methods used: 



(i) At the beginning of the course: Studenty are/painedand/a^hed/to 
'interview' eachother to* find/ out:- 

0 why the/ student enrolled/ for the/ course/ and/ what they hope, to- get 
from/ the/ counie/ 

® wh^pre^U>U4c crperience/ the/ student ha^ of Shiatyu/ tr any other 
ader native therapy and/ whether they hold/ any qiAaltfuzatioviy wv 
orthodox/ or alternative/ medicine/. 

They iy then/ relayed/ to- the/ whole group yothatl<nowledsge/ eon/ he shared/ 
and^^udenty feel they know eaeh other. 1 checks ^tudenty medical history 
for any cont r ain<licevtio^ vxeed/tob&tdken/ CntoaecouA^t w 

ShiatiM/ practice/. 

(ii) During the course: StudentypraetUe/giA/Cn^ShiatyA/in/th^ 

pohtion&r and/ 1 go-round/, observing' their p raettce/ and/ gUvin/g' adA/iee/ on/ 
their individual practices. 1 ado cdloccvte/ 1 5 minuteyper meeting for 3 
indtA/id/Aaltidorial^ while/ the/ red: of the group c^re/provctXdrug'. I 
encovcrage/ ytudenty to corUXnualLy eA/oduate their ow w postures and/ 
comfort while/ worlUng/and/tOadkfor feedbtAeh from/ th^ f client when/ 
they give 5 hiatyA/ treatment, 

(iii) At the end of the course: Studenty give/ a/ whole body ShCatyv to their 
partner, which I observe and/ comment cm, I incite/ dxdenty to s,ay what 
they feel they hoA/e^ got out of the/ course/. We/ then/ go through the/ vcvriouy 
opttony for further learnings (n/Shiatyu/ or other heedih courses of^ 

M any Ward/ and/ elsewhere, depending' on the/ studenty interedy. I 
covvdAAetthiA a& a/ two-way guedtonfr anywer ^e^don with the/ whole/ 
group. 



2. Background information on student, including previous learning experience 
and aspirations 

Student hi o^. 2 came to the course with/ yyyr^^ny^led^ of body worh 
haainfr gained/ bowie experience of bites muyycage/. She way in/edtguting/ 
Shuitbatoifee/what wwinwlved in treatment and how thiy would/ uUt 
her ay an addttXonal/ therapy. She/ ha# had/ nofbrmcd training (n 
orthodox medicine/ or aXternatt^e medicine . 



3. "Distance Travelled’ 

i.e. Student's progress as measured against course learning objectives (to be attached) 



Student 2 way keen to learn new technique#, poyytbly at the expense of 
mjayterinfr thoye ad'eady taught, hut dne/W a* keen and confident fa 

practice/ outride/ the/ teaching' ypace and/ would/ give/ her partner 
Shiatya treatment at home/ between seyyCony. He#- enthu^inymtoprogre^ 
way an area in which/ the/ needed/ to learn home/ patience/, yince it resulted/ 
in practice/ which/ way not aUvayy appropriately paced/ for the/ recipient. 
That yaid, student 2 way qutch to interpret the uye of the techniquey^ 
adapt them/ to her own particular treatment and/ ytyle. 

Student 2 caneaydy and safely uttU&e the/ new techniquey learned/ within 
her ovwyite murage/ treatmenty. 



4. Additional benefits of learning 

i.e. other outcomes, not included in the learning objectives, that are gained by the 
student from the course: e.g. personal benefits (growth, development, confidence, 
initiative, motivation), economic (career progression or development) or interpersonal 
(negotiation, collaboration, interaction) 

Student 2 found being' in a group with a mature/ of agey and ahidUtey 
yjrnethincp of a challenge/, hut she/ coped wed and gained greatly in yelf- 
confidence and became/ more ykilded/ in yociai interaction from the 
experience. 



5, Advice to student on ‘next step’ 



My adaice/for thty ytudent way that if Shiatyu were tobecomepart of her 
further career development, yhe yhould enrol/ on aprcudtticrner diploma/ 
course/, where the/ could/ fully experience/ Shlatya and make/ it port of her 
fourudatuony in hodyworh. Thiy would strengthen her whole understanding 
of body energetCcy. 1 would encourage her to try out Shiatya ychoohy 
practice cdayyey to find which/ ychood yhe feedy moyt comfortable with 
continue/ her learning' whllyt making/ thiy selection. 
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Mary Ward Centre 

Moderator’s Report Form for Assessment of Learning 



Appendix 4 



Course Tutor’s Name: T. T 

Course Code and Title: ..559 ShXxztbu/ 

Tutor’s Assessment: Case 1:1; Case 2:2; Case 3:3 

Moderator’s name: M- h) 

Date of Moderation:.. .10/1/01 



1 



ASSESSMENT METHODOLOGY 



1 . Method of initial assessment was 
effective in establishing entry level 


✓ 








Initial assessment method did not establish 
student level 


2. On-going assessment methods were 
appropriate and regular 




s 






On-going assessment methods were 
inappropriate and irregular 


3. Method of Final assessment was 
appropriate 


✓ 








Final assessment methods were 
inappropriate 


STUDENT ACHIEVEMENT 


4. Judgement of student progress was 
fair and appropriate 




s 






Judgement of student progress was 
inappropriate 


5. Judgements were clearly made 
against the learning objectives 










Judgements did not refer to learning 
objectives 


6. Judgements considered additional 
benefits of learning 




s 






Judgements did not consider additional 
benefits of learning 


7. Evidence given to support 
judgements 










— — Q, 

No evidence given to support judgements 


FURTHER PROGRESSION 


a. Advice given to student about their 
next step was clear 


s 








Advice on progression was unclear 


MAK1NC USE OF THE ASSESSMENT 


V. Assessment was clearly used to 
monitor student progress 


s 








Assessment not used to monitor progress 


10. Assessment used to give feedback 
to students 


s 








Assessment not used to give feedback 


1 1 . The student was involved in the 
assessment process 




s 






The student was not involved in the 
assessment process 


12. Assessment used by tutor to 
evaluate and alter course 




s 






Assessment not used by tutor to evaluate 
or alter course 



! GRADE DESCRIPTORS 


1 


Excellent: a model for others to follow 


2 


Good: strong in this area 


3 


Satisfactory: a minimum standard which we would 
hope to see bettered 


4 


Less than satisfactory 


2 


3 4 



12. Please make any further comments here on the effectiveness and appropriateness of 
the methodology and judgements 

r. r. ^particularly %oo^aZ wui 

other tutory could, learn, from, hi* approach,. Hi* integration, of 

(yV\to~ th& covLv~b€/ ifr (fa\o£h&y’ 

I wouM,ltUto-iee,a,ck>ier e*aluxxtUm,ag<xtn*tthe,4- couth* objective*. 



1 3 . From the information you have, where would you place each of the three students 
presented in the case studies? (For tutor’s assessments, see top.) 



— 7^\ 


Example 1 
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THE MARY WARD CENTRE 



APPENDIX 5 



How the Mary Ward Assessment Strategy matches the Common Inspection 
Framework for inspecting post-16 education 



Common Inspection Framework 
criteria for effective assessment: 


Evidence in the MWC Strategy of 
effective assessment 


® forms of assessment and recording 
are suitable for courses 


« Moderator Report 
® Verifier’s Report 
® Departmental Self-Assessment 
Report 

® Examples of student assessments 


® assessment is fair, accurate and 
regular 


® Moderator Report 
® Verifier's Report 
® Examples of student assessments 


» initial assessment is effective 


® Moderator Report 
® Verifier’s Report 
o Examples of student assessments 


® assessment is used to monitor 
progress 


® Tutor Course Report 
® Examples of student assessments 


• assessment is used to inform learners 
about progress 


« Moderator Report 
• Examples of student assessments 


• achievements towards learning goals 
are recorded 


• Tutor Course Report 

* Examples of student assessments 


• assessment information is used in 
programme development 


• Tutor Course Report 
® Moderator Report 

• Verifier's Report 

• Departmental Self-Assessment 
Report 


® assessment procedures follow 
regulatory body requirements 


® n.a. for non-accredited courses 


* those with a legitimate interest, e g. 
employers or parents, are informed of 
learner's progress 


® n.a. for adults not sent by employers 



PARK LAME COLLEGE, LEEDS 
The Stepping Out Programme 



Philippa Lester and Phill Hopkins 



1 . CONTEXT 

1.1 Park Lane Coliege is a large general education college in Leeds. The 
Department of Adult and Community Education had over 12,000 
students in the year 1999/00 and offers a wide ranging programme 
focussed on widening participation and enabling adults to access new 
opportunities in education and training. 

The work of the Department of Adult and Community Education is 
funded by Non Schedule 2 (NS2) through the local authority, and other 
sources such as European Social Fund and the Basic Skills Agency and 
Adult and Community Learning Fund grants. 

In the year 1999/00 the college received £140,000 for the delivery of its 
Non Schedule 2 provision in partnership with Leeds City Council. 
£30,000 of this sum was allocated to working with adults with mental 
health difficulties through the Stepping Out Programme. 



1 .2 The Stepping Out Programme - Learning Opportunities for Adults with 
Mental Health Needs, provides over 30 part-time courses city-wide at 
12 different venues, including day hospitals and centres, community 
centres and college centres. Courses last from between 10 and 30 
weeks each academic year, depending on the particular needs of the 
learners. In 1 999/00 38% of the courses were Schedule 2, leading to 
formal accreditation and 62% were Non Schedule 2, unaccredited. 

In the same year there were 224 student enrolments of which 52% 
were female and 48% were male. 

The present curriculum offer consists of: Basic Skills - Literacy, 
Numeracy, Confidence Building, Speaking and Listening, Study Skills 
and Information Technology; Animal Care, Creative Writing, Pottery, 
Cookery, Art, Aromatherapy, Woodwork, Sculpture. The staff team is 
made up of a full-time programme manager and eight part-time tutors, 
including a basic skills organiser, who teach between two and eight 
hours each week. 
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2 . 



LEARNING OUTCOMES MODEL 



2.1 Scheme of Work 

e All tutors, whether teaching an accredited or unaccredited course, 
produce a general scheme of work for each of the courses that 
they deliver 

• Each scheme is designed with the assistance and specialist 
knowledge of the programme manager, using prior knowledge 
and information that has been gathered via formal and informal 
discussions with current and prospective learners, and 
professionals/agencies working with this particular client group. 

® The schemes are directed at both the needs of the individual 
and those of the group. 

2.2 Learning plans 

• Within the first half term of a course the tutor will negotiate, within the 
framework of the general scheme of work, an individual learning plan 
with each learner 1 . 

® Tutors and learners complete a standard pro-forma to monitor 
progress. 

e Tutor is encouraged to help learners to devise alternative means of 
target setting and record keeping. 

• Students are encouraged to develop ownership of their learning. 

® Students are encouraged to identify and work towards their own 
individual learning goals as well as meet the programme and course 
learning outcomes 

N.B There my be a tension between the empowering experience of 
students having control over their own programme of study, and 
involving essential paperwork. 

3. DEVELOPMENT OF MODEL 

3.1 The programme has its own steering group that is made up of senior 
management from the various mental health services, agencies and 
organisations; the programme manager and senior departmental staff. 
The programme manager also represents the college locally as a 
leading member of the Education & Training Sub-group of the Leeds 
Mental Health and Employment Consortium and nationally as the 
convenor of the SKILL (National Bureau for Students with Disabilities) 
Mental Health Special Interest Group. 
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3.2 The development of individual learning outcomes 

On NS2 unaccredited courses individual learning outcomes can be 
very wide ranging. Subject based outcomes in the form of ‘products’, 
such as personal project based work, e.g. in a pottery workshop, one 
student’s desire was to “ make a dish and decorate with slip”. Whilst one 
student's target during a term was to work towards recording, producing 
and distributing a CD of his own writings and piano performances. 
Learners have also identified outcomes in regard to progression into 
the mainstream provision. Here a student recounts her experience 
and increased confidence gained during an unaccredited creative 
writing course: 

“(The) atmosphere made it feel comfortable to test myself out with reading out 
loud things I had written, knowing that it would be okay to make a mistake, and 
then okay to try again. Without the pressure of feeling a failure or being left behind, 
it seemed a natural progression to become more confident with my efforts at 
writing creatively and enjoying it. ” 



Then she describes her experiences of progressing beyond 
the course: 

“With the feeling of success in this group, I felt as though I could try to step out of 
the day centre setting into a creative writing course 'outside' in the community.../ 
have recently started a W.E.A. course on Creative Writing, and now feel, rather 
nervously, included in their group. It feels a big step to take but, preparing for it 
first, by being introduced to the subject, on my own terms and my own ground, 
has definitely made it feel possible’’ 



Outcomes concerning personal progression pathways can be illustrated 
by the tracking of an individual learner's progression, in this case, over 
a two-year period. 



1 0 week 




30 week 




30 week 


unaccredited 




unaccredited 




accredited (S2) 


(NS2) 




(NS2) pottery 




OCN pottery- 


aromatherapy 


— > 


workshop 


-> 


course 


course 




at a mental 




at a mainstream 


at an acute 




health day 




college centre 


menial health 




centre 






day hospital 











The above illustrates not only an improvement in an individual’s state 
of mental health, but more significantly here, the benefits from 
individual learning outcomes being more than just subject based and 
allowing the learner to develop their own learning pace. The 10 week 
aromatherapy course was not only seen and used by the learner as an 
end in itself, but as an introduction to the rest of the programme and 
then the college as a whole. 
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This is what the learner’s spouse has said: 

“(Name) needed pushing., he lacked so much confidence to go onto other courses. 

He’s always been interested in art and we are ever so pleased that he can now 
move on" 

3.21 Sometimes a learner may only be able to identify that they have met 
learning outcomes in retrospect. This can be as a result of periods of 
memory loss due to symptoms of a particular illness or from the varying 
side effects of medication. With the help and guidance of the tutor the 
learner may need to excavate positive aspects of their learning when 
things aren’t going so well and from their view point, they have made 
little progress. This is where the Student Learning Diaries play an 
important role. They can be used to illustrate progress (and the lack of it) 
during the course and help a learner to see areas of success, as well as 
those areas of study that need to be re-addressed. 

3.22 For some learners who have identified a personal outcome to progress 
into the mainstream provision, it may also be important, and often vital 
to their success, that they remain in contact with their Stepping Out 
Programme tutor. There are a number of learners who have enrolled on 
mainstream courses, often at a college centre near to where they live, 
who continue, for varying lengths of time, to attend a Stepping Out 
Programme course. In these situations the learner, the tutor and 
sometimes support staff from the mental health services, can all work 
together to help the learner focus on the areas of difficulty that they 
continue to encounter. This learner has progressed in to the mainstream 
but continues to access the programme: 

“The class has helped me greatly to gain confidence, first in a protected 
environment and then hopefully to more mainstream education. I am already 
studying French at Park Lane. I still find this difficult but hope it will get easier. ” 

3.23 The majority of the programme learners are generally attracted to the 
NS2 courses because they do not have the added pressure of, what 
learners understand to be, formal qualifications. But this is not an excuse 
for the learning outcomes of a course to be vague, imprecise and 
inaccessible to the learner. It is vitally important that the tutor, when 
necessary, is explicit in their description of the aims and objectives that 
are attached to the course. For this reason it is important that the course 
has a definite structure for those learners to adopt if they do not have 
the confidence and ability to develop their own lines of enquiry. Often a 
learner will drift in and out of a course if they feel insecure about their 
own learning and progress. This is where an Individual Learning Plan is 
important as it can translate the often jargonistic language of a tutor’s 
general scheme of work in to terms that can then be understood and 
owned. When visiting individual classes, where this infrastructure is in 
place, one can immediately sense the buzz of learning of a group who 
are on track. 
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3.24 With all these things in mind though, some learners will continue to 
enjoy the added kudos of gaining a formal qualification, as this learner 
simply puts: 

“I wish we were working towards some certificate’. 

3.3 Group learning outcomes 

Group learning outcomes can often be devised by the use of a Group 
Learning Agreement. Learners can agree to specific outcomes that they 
would want to meet at different periods during or by the end of a course. 
These are one group's comments recorded when reviewing their 
learning: 

“We have gained confidence to write something and then feel able to read it out 
loud. We supported each other in our writing, with ideas, feedback and artistic 
criticism. We have gained interests. We have entertained others. We have achieved 
a lot and have more conMence.. being able to communicate with less 
embarrassment at lack of education". 

3.31 A group of learners at a day hospital have collaborated for the past three 
years in the production of a high quality full-colour calendar. The 
production process involved a great deal of individual and group work. 
Each learner developed and then produced a piece of artwork. The 
group then had to make decisions regarding selection, design and 
layout, costing and printing, and then the distribution of the finished 
product. The third calendar produced also involved collaborating with a 
group of learners from a literacy course, in the production of pieces of 
prose to accompany photographs. This year a group of learners from the 
programme have been nominated for an Adult Learners' Week Award. 
The goals that the group has identified include: 

• Ensure creativity and self-expression. 

» Help students returning to learning by improving their skills in writing 
and discussion. 

• Enable members to enjoy writing their own work and reading that of 
established authors" 

Many projects were identified, including: 

» Performance of own work at retirement celebration of centre worker 
» Write and perform play at centre open day 
» Stage exhibition of creative writing 

» Take part in training day with social services staff for college tutors 

The group also highlighted the following as being some of their collective 
outcomes: 

» The audience at the open day was much larger than we expected. 
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© The group has also collaborated with the Sculpture and IT classes at 
the centre, writing material to stimulate artwork and for use at the open 
day in the form of captions and headings. 

® The trip to France and Germany was very successful, organised by 
one member with those travelling helping each other. 

® Several individuals have obtained publication in newsletters and one 
in the Radio Times. 

3.4 Monitoring Learning Outcomes 

Outcome data is tracked both internally from within the programme and 
externally. At the Programme Staff Team meetings tutors are able to 
discuss and compare the types of experiences and learning outcomes 
that learners have met, or are working towards at any given time. There 
are regular celebrations in recognition and to highlight both individual 
and group successes to family members, friends, peers, and other 
professionals working in the field. 

The programme manager and administrative staff track progression data 
on a database to research into the numbers of learners moving onto 
further provision. The programme manager maintains links with other 
departmental colleagues at half termly programme managers meetings 
and on a less formal basis by visiting other college centres. These 
relationships have proved vital when setting up new courses. 

The programme is subject to standard college monitoring procedures. 
These include mid year and end of year reporting on retention, 
achievement, and progression and triennial programme reviews. 

As well as recording individual outcomes and tracking this data, the 
programme is inspected externally by Leeds City Council who monitor 
attendance figures on a weekly basis and make observation visits to 
selected courses annually. 

4. STRENGTHS 

4.1 The development of the unaccredited part of the provision where 
an individual can start to (re)discover how best they learn. 

4.2 Students have been able to develop competence in the following areas: 

® To become comfortable and feel at ease in the ‘classroom’ 
or group situation. 

® To develop necessary and particular social skills that are needed 
for such an environment. 

® To develop study skills, such as concentration, time keeping and 
portfolio building/organisation. 

• To build confidence and improve self esteem. 

• To develop specific skills associated with the subject, for example, 
pottery. 
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4.3 Learners have acknowledged the importance of these particular kinds 
of skills. Their comments include: 

“i Being embarrassed to share work or express an opinion because lack of 
experience in a group” 

“ Gained in confidence and understanding " 

“Had fun mish-mashing ideas and learning new ways of writing" 

u Being in a group gives me ideas " 

w Getting to know members more cioseiy and understand them ” 

1 made a ding-film dish . Glazed my dish . Enjoyed every minute. Had a good 
morning with determination to make something good " 

u This class stimulates the thinking and brain and we try to concentrate for the 
two hours we are there" 

4.4 Bite sized chunks of learning. 

4.5 Established progression routes from NS2 to S2 provision. 

4.6 Flexible approach to programme planning and organisation. 

4.7 Programme is fully embedded within the department of Adult and 
Community Education. 

4.8 Programme is held in high regard by all of the major mental health 
service 

agencies and organisations in Leeds and has attracted sponsorship. 

4.9 Positive community profile which includes: 

• Adult Learners’ Week events. 

• Creative writing publications. 

• Open days and art exhibitions. 

• Poetry recitals and play readings. 

5. WEAKNESSES AND AREAS FOR DEVELOPMENT 

5.1 Tension between the empowering experience of students having control 
over their own programme of study, and involving them in essential 
paperwork. 

5.2 A citywide council questionnaire for all learners enrolled on NS2 courses 
identified a need to improve information on the opportunities available 

to them when the course has finished. 2 
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6 . 



NEXT STEPS 



S.1 Three types of post course progression information have been identified 

that will compliment that which is already available: 

® Progression opportunity citywide connected to the course subject. 

® Local progression opportunities within easy access of a particular 
centre(s). 

® Tailor-made information sheets for individual learners with particular 
needs, for example, college centres that are located on a particular 
bus route. 



NOTES 

1 . The individual needs of a learner are no different whether they are enrolled on a 

S2 or NS2 course and their experience of any of the programmes courses should be that 
there is no difference between the two, apart from accredited outcomes. 

It should be seen as a seamless provision. 

2. Within the programme, and of those surveyed, to this particular question they answered, 
12% said 'definitely' (department 16%), 26% said 'mainly' (department 14%), 38% said 
'partly' (department 28%) and 21% said 'not at all’ (department 22%). 
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BEST COPY AVAILABLE 



WIGAN AND LEIGH COLLEGE 



BermacSetie Georg® 



1 . CONTEXT 

1 .1 Wigan and Leigh College is one of the largest Further Education 
Colleges in the country with approximately 27,000 enrolments during 
1999-2000, of whom 5,000 were full-time students. Just fewer than 8,000 
adult part-time students followed their studies in over 80 community 
venues across the Borough. 

1 .2 The College itself comprises four main sites within the Borough of 
Wigan, an area of over 300,000 people within two towns, Wigan and 
Leigh, and their surrounding districts. It is approximately 20 miles 
equidistant from the cities of Manchester and Liverpool. 

1 .3 The College has a regional catchment area for its specialist vocational 
programmes, part-time management provision and for most higher 
education programmes and a national and international client group for 
consultancy work, distance learning programmes and for collaborative 
provision. 

1.4 Adult outreach provision is largely undertaken through College in the 
Community located within the faculty of Access and Continuing 
Education. It was created in 1995 to manage a partnership arrangement 
with fourteen high schools across the Borough; further partnerships were 
formed by extending links with other agencies, both public and private. 
For example, Family Learning and Basic Skills programmes enabled 
partnerships to be formed with numerous primary schools. 

1.5 A wide range of Schedule 2 courses from GCE A’ Levels to Accredited 
Vocational courses are delivered across nine faculty areas alongside a 
broad non-vocational programme in a variety of venues. 

2. THE LEARNING OUTCOMES MODEL 

2.1 Students 

People from all eleven socially excluded groups outlined in the initial 
FEFC circular 99/16 inviting bids for NS2 projects were targeted 
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2.2 Categories of outcomes 



® intended course outcomes - the agreed subject specific learning 
outcomes 

e individual learning outcomes - personal to the individual learner, 
such as confidence building. 

2.3 Staff 

® subject tutors 
® learning facilitators. 

2.4 Identification of objectives 

® Tutor for the first subject choice meets the group and negotiates an 
agreed set of learning outcomes, which are specific to that subject. 
Tutor is expected to use his/her expertise and experience as a guide 
for identifying and setting aims and objectives, for example in 
computing learners might have nothing more specific than the single 
aim of using a computer. 

® Tutor and learners log the agreed learning outcomes on separate 
paperwork: 

o The Sutor record • a simple assessment grid for every learner 
with general learning outcomes, which are very specific and 
fundamental to the concept of learner involvement and 
progression and remain the same across the range of subjects 
offered under. 

o The learner’s record - a log of personal as well as subject 
specific learning outcomes under the general heading ‘What I 
want to Learn' 1 (Appendix I - examples of completed forms) 

• A learning facilitator visits each group during the second or third 
session to assist learners complete an individual action plan, 
encouraging learners to think about the subject choices they have 
made, why they were made, and begin to formulate goals. (Appendix 
II - example of Action Plan) 



2.5 Recognising learning progress 

® Learning progress within the each subject is assessed by the tutor 
against the general and specific learning outcomes, and by the 
learners against their subject specific and personal learning 
objectives. 
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e Assessment of ‘added value 1 and ‘distance travelled' identified 
through 'Managing Learning’, which encourages self-assessment 
and an evaluation of the learning process (see development section). 



2.6 Sources of evidence 



© 



© 

© 

© 

© 



student self assessment 
direct tutor observation 
audio tapes and video recordings 
photographs 

examples of written work 
examples of practical work 
computer disc 
peer assessment 



2.7 On course qualify monitoring 

® Weekly review by NS2 manager 

e Regular report made to Head of Faculty, Director of Curriculum 
and NS2 Co-ordinator for the Wigan Lifelong Learning Partnership 

® Regular meetings to review quality of provision with managers, 
tutors and learning facilitators. 

® Visits to groups by learning facilitators and NS2 Manager 
for feedback. 



2.8 End of course review 

® Learners' review of action plans supported by learning facilitator. 

® Progression information disseminated by learning facilitator. 

® Evidence of assessment provided by tutors via learning 
outcomes data and a sample of learners’ work. 

® Attendance registers analysed 

® ‘Making Choices/Managing Learning’ documentation reviewed 
including individual student action plans and evaluation forms 

® Information reviewed on progression to a more advanced level 
course by individual learners or the group 4 . 

® Database and Spreadsheet information on retention, achievement 
and progression analysed at group and individual learner levels. 
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2.9 Recognition off student achievement 

© Certificate awarded itemising the programme studied. 



3. DEVELOPMENT OF THE MODEL 

3.1 The College in the Community team at Wigan and Leigh College, with its 
extensive community partnerships, undertook the work on learning 
outcomes as part of the FEFC non-schedule 2 pilot project. 

3.2 During 1999-2000, approximately 1 ,000 students took part in non- 
schedule 2 courses, choosing learning programmes of between 30-60 
hours. Programmes were not usually delivered as single subjects. 
Learners were encouraged in group discussions to select at least 3 
subjects from a menu of choices and to negotiate an appropriate day, 
time and venue, since a major objective of the project was to develop 
and offer a range of quality, learner-centred and accessible programmes 
delivered in the methods best suited to the needs of the target groups, 
at times and places best suited to them. 

3.3 In order to give a coherent framework to the whole concept, 2 National 
Open College Network courses which had been produced in response 
to the Kennedy Report on Widening Participation were used 1 : 
(Appendix III - Making Choices/Managing Learning paperwork) 

» 'Making Choices' - for the initial negotiation to set up learning 
programmes 

• 'Managing Learning’ - to enable learners to evaluate the learning 
process itself. 

3.4 Staff development was undertaken to train tutors in: 

• Negotiation of learning outcomes with different types of learners 

• Identification of learning progress against target outcomes 

• Methods of delivery in non-accredited learning 

3.5 With the exception of an advocacy course for visually impaired people 
who required specially adapted computer equipment and a welding 
course for a group of travellers, which for health and safety reasons had 
to take place within the College, all NS2 programmes took place in 
community venues. 

3.6 Care was taken to ensure that all the services enjoyed by students 
within the College were extended to NS2 learners within the community. 
Tutorial support was extended by the innovation of learning facilitators. 2 





3.7 The project’s work on learning outcomes evolved with the support of 
information meetings led by the FEDA. Ideas were formulated on paper 
and presented to the tutor team. Once the process was initiated further 
meetings took place to evaluate the process with follow-up visits by the 
NS2 Manager to various community groups. 

3.8 The process was adapted in response to regularly sought 
tutor/learner/learning facilitator’s evaluation. 

4. STRENGTHS 

• Learner participation in setting learning goals and measuring own 
achievement 

• Learners’ achievement recognised through attaining personal 
and subject outcomes, certification and progression. 

» Listening to and responding to the voice of the learner through 
self-assessment and subject evaluation. 

» Tutors thinking less in terms of a syllabus and more in terms of 
negotiating a learning programme which is responsive the learners 
motivations and needs 

» Tutors devising ways of teaching and assessing outcomes in ways, 
which accommodate different levels of student learning. 

® Tutors more aware of the learning process through mapping 
achievement. 

» Staff development and a stimulating exchange of ideas 
between peers. 

• External commendation for learning facilitator initiative 



5. WEAKNESSES AND AREAS OF DIFFICULTY 

® Paperwork not necessarily tutor or learner friendly - use of 
education jargon 

• The concept of negotiating learning outcomes bewildering for 
some learners with negative previous experiences of education 

» Unfamiliarity of new learners with the concept of self-assessment. 
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© Unfamiliarity of new tutors with the concept of negotiating learning 
outcomes with apprehensive learners - often requiring ‘on the job' 
staff development 

© Lack of understanding by new tutors of the need for assessment 
in non-accredited courses. Much staff development required, 
particularly on assessment methodology and what counts as 
acceptable evidence. 

® Personal outcomes such as confidence building and self esteem 
difficult to assess 

® Paperwork sometimes seen as intrusive to teaching and learning 
and required careful introduction 

® Some subject tutors completed the paperwork on behalf of all their 
learners, particularly in mental health groups, rather than working 
with the study support tutors. 

© Producing documentation which is learner held and of value in 
mapping individual progress and achievement. 

» Identifying what counts as acceptable evidence of achievement and 
distance travelled’ for inspectors and funders as well as students 



S. CONCLUSIONS AND NEXT STEPS 

6.1 During this academic year, tutors have been visited within the learning 
situation to gauge their response to the learning outcomes paperwork. 
Further changes were made in the light of those responses. 



NOTES 

1 . Discussions were held with Greater Manchester Open College Network to gain their 
permission to adapt the basic concepts of the courses to suit the learning needs of non- 
schedule 2 learners. 

2. Learning facilitators were a feature of provision, which was favourably commented on by 
the FEDA/NIACE evaluators and by Learning Northwest. The facilitators are former 
students who draw on their own experience to encourage people to return to education. 
They mentor individuals, help with action planning and feedback to the college co- 
ordinator. 

3. The initial heading Negotiated Learning Outcomes was felt to be inappropriate on a 
form retained by learners, on which they logged their own progress. 

4. Students on the programme progressed to courses which include: City and Guilds 4242 
Basic Competency in IT from NS2 Computers Made Easy, or NCFE Practical Craft Skills / 
Mixed Crafts from NS2 Mixed Crafts, or Family Learning from NS2 Introduction to Family 
Learning etc. 
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WIGAN a LEIGH 



APPENDIX 1 



Learning Outcomes for Non-Scheduile 2 Provision 

Learner’s Name: A.Student 

Subject: Computers Mode Easy 

Tutor: A. Tutor 

Centre: Community Centre 

Date: September 2000 



Negotiated Learning Outcomes: 


1 can do 


1 am working 
towards this 


1 cannot 
do this 


I can plug in and switch on computer 
I can get to my file 
I can switch off and close down 


/ 


Not confident 




I am learning spell check 
I feel more confident 




✓ 




I can do basic word processing 
I am not confident with spreadsheets 


✓ 






I can scroll up and down 
I have learnt to open file and use ’ tools' 
I can save my work 


✓ 




✓ 


I feel much more confident 
I can size and alter margins 
I can embolden 








I can use cut and paste 
I can open and close files 


/ 







Signature: A. Student I think that I hove accomplished more than I realise 

(Student) I feel much more confident now , but I sometimes have to 

play about' to achieve my objective. 




11 



83 



WIGAN & LEIGH 
APPENDIX I 



Non-Schedule 2 Programme 
To be completed by tutors for Individual learners 



Lean 

Lean 


ung Outcomes negotiated with the 
ler 


Yes 

/No 


Evidence (Assessment Activity) 


Yes 

/No 




Global: 








Yes 


1*1 


Take control of own learning 


Yes 


1.1 


Negotiate learning outcomes and 
complete an action plan. 




1.2 


Work to an appropriate standard 


Yes 


1.2 


Direct tutor observation 


Yes 


1.3 


Demonstrate requisite competence 


Yes 


1.3 


Audio tapes of discussions, photographs of 
activities \ peer assessment, video evidence of 
practical demonstrations, saving materia! on 
disk. 


Yes 


1.4 


Make significant progress towards 
fulfilling goals and po ten tiai 


Yes 


1.4 


1, 2 and 3 above 


Yes 


1.5 


Nego tia ted L earning 
outcomes: 


Yes 




An ability to discuss, reflect and consider 
other students thoughts and ideas. 


YeT 




Developed skills in group works and 
decision making. 








1 .6 


Demonstrated an ability to discuss 
ideas. 


Yes 




An ability to debate effectively 


Yes 


1 .7 


Developed ideas into practical activity 


Yes 




Explain thoughts and work out, how to put them 
Into practise. 


YeT 


1.8 


Increased awareness of colour mixing 
and matching 


Yes 




Describe complementary colours 


Yes 


1 .9 


Demonstrated an ability to apply 
emulsion and acrylic paint 


Yes 




Explain what materials/tools are needed to 
apply paint 


Yes 


1 .10 


Recognised the difference between " 

stencils & templates. 


Yes 




Describe how to make stencils and templates 


Yes 


1.11 


Developed confidence in the practice 
applications of decorating materials/ 
tools 


Yes 




List in order of importance the basic steps used 
when decorating a room. 


Yes 



Further Comments 

^ u * or A.Tutor 

L ®amer A.Student 

Learning Programme Interior Design (5 weeks) 

Vonue Social CJub i4 
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WIGAN & LEIGH 
APPENDIX II 



PERSONAL ACTION PLAN 



Name:- 



Date:- 



Centre:* 



What do you hope to achieve through the choices you have made? 



What factors may affect your personal goals e.g. family and other responsibilities; costs and other 
financial issues; travel, timescales; help needed with study or basic skills? 



How could you achieve your personal goals e.g. information on courses; support from College services 
such as study or basic skills; more courses at a higher level at your centre? 



Further comments 



Signed - Signed:- 

f Student] (Tutor) 



C:drive/Ahayes/Presentations/App20.03 



Name: Date: / / 

Centre: Learning Programme: 



The Method of Study: Group discussions 

Teacher's notes 
Gathering information 
Teacher/group negotiation 
Teacher demonstrations 
One to one instruction 
Other 



Which methods have been the most useful to your 

learning progress? One to one teacher help 

Teacher led lessons 
Group work 



Following the learning Programme 

has the information been: Clear 

Relevant 

Delivered in a logical order 



Learning Process: 

Have there been any difficulties with following the 
learning programme? 

What would help to achieve your learning goals? 



Yes/No 

Help with English 
Help with Maths 
Better location 
Better facilities 

A longer learning programme 
A shorter learning programme 



Identify a learning achievement within the 
learning programme 
identify a new skill 
identify a new interest: 



Personal Goafs: 

What personal goals are being achieved through 
the learning programme? 
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THE WORKERS EDUCATIONAL ASSOCIIATJON 



Philippa Langtorn 



1. CONTEXT 

1.1 The Workers’ Educational Association (WEA) is the largest voluntary 
provider of Adult Education in the United Kingdom. It is organised 

in England into 13 districts and in Scotland through the Scottish 
Association. Wales and Northern Ireland have their own independent 
WEA structures 

1.2 WEA learners vary enormously in age, background and interests and 
so do their learning needs. It is the WEA's aim to promote and deliver 
high quality learning, which is responsive and relevant to all these adults. 
In line with this, the WEA promotes three main course programmes, 
which continue to diversify in response to new needs and initiatives: 

• The general education programme, open to all adults, now includes 

a far wider range of science and technology courses as well as many 
more courses with a distinct practical element to them than was the 
case even five years ago. 

® Community learning focuses on the needs of particular groups in 
the community and now makes up the second largest and fastest 
growing part of the WEA's overall provision 

• The WEA’s workplace learning programme is particularly concerned 
with the needs of low paid workers most of whom left school at the 
minimum age with few if any qualifications. This programme is also 
growing in size and becoming more diversified. 

1 .3 While the proportion of externally accredited courses within the WEA 
continues to rise, a significant number of courses in both the general 
and the community learning programmes are not subject to external 
assessment for sound, educational reasons (1). Since the mid-90s these 
courses have been subject to the WEA's Learning Outcomes process, 
which enables individual learners to reflect on their own learning aims 
and learning achievements. From the outset the application of Learning 
Outcomes in the WEA has been seen as a way of validating the learning 
experience of learners; it is not an attempt to quantitatively measure 
learning. 
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1 .4 The WEA is one of a number of institutions, which received FEFC 
funding for non-schedule 2 work. Issues of quality assurance and 
student learning gain from its non-accredited programmes have therefore 
been a focus for development for several years. 

2. LEARNING OUTCOMES MODEL - WEA London District for the 
Generali Programme 

2.1 Process (Appendix I - Tutors' briefing) 

While the WEA’s Learning Outcomes process is applied across the 
whole of the Association, there have been slight differences in the 
practices applied by different WEA Districts. The procedure followed by 
the London District is described below. It takes into account the fact that 
there are specialist members of staff in London District responsible for 
different areas of the curriculum and that these members of staff are 
organised into three programme teams: 

Step 1 : Tutors submit a course outline for approval by the Subject 
Officer prior to the course. This is brief, includes Learning Outcomes, 
and gives some indication of proposed teaching methods (Appendix II - 
completed course outline). 

Step 2: At the first meeting of the class, the tutor discusses and agrees 
the Course Outline, including the Learning Outcomes, with the learners, 
and explains that at the end of the course they will be asked to describe 
what they judge they have learned and achieved. Amendments and 
additions may be made to the Course Outline at this stage. 

Step 3: At the start of the course the tutor also receives copies of the 
Student Learning Outcomes Report that has the agreed Learning 
Outcomes on it. Tutors are responsible for making learners aware of this 
and distributing it for completion at or towards the end of the course 
(Appendix III - completed student report). 

Step 4: The tutor also receives a Tutor Report proforma. The tutor 
reads all the Student Learning Outcomes Reports and: 

® Describes and evaluates the students’ learning and achievement 
generally using the learners’ self-descriptions and their own 
perception as the teacher. 

® Evaluates the course as a whole using the learners' judgement of the 
course process, together with their own evaluation of the course. 

Step 5: There are two key stages: 

» The Subject Officer reads all the Student and tutor reports for 
each course, and identifies any points for action 
® The Subject Officer produces an overall report for his/her subject 
area, together with key recommendations, for review, discussion 
and action at Team and at District Committee. 
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3. DEVELOPMENT OF THE MODEL 

3.1 In 1993 the WEA London District and the Working Men's College 
convened a conference on how to describe quality learning on non- 
accredited courses as a way of evolving a strategy for measuring 
learning outcomes which was fit for purpose, rather than waiting for 
one to be imposed from the outside. This was followed by a period of 
intensive developmental and design work which culminated in a number 
of detailed Learning Outcomes pilots which took place in both rural and 
urban settings, but which were limited to the Association's branch-based 
general programme. 

3.2 The WEA’s three year National Strategic Plan launched in 1995 
contained the objective that Learning Outcomes would be applied 
throughout the general programme and would be developed for the 
non-accredited part of the community education programme by 1998. 
This was achieved. 

3.3 In 1999 a national seminar was held to appraise the Learning Outcomes 
strategy to date. This took into account: 

» The experience gained during the past five years 

o The views expressed by the FEFC Inspectorate following 
inspections of the WEA in 1995 and 1999 

» The new post- 18 educational world 

3.4 On the whole, it was felt that the experience of the past five years, 
especially in relation to the general programme, confirmed the validity 
of the approach taken. 



4. STRENGTHS 

• Student evaluations showed that they had benefited from the clearer 
course outlines and proposed learning outcomes, which had been 
prepared by tutors trained and supported in this approach. 

s FEFC inspectors welcomed the initiative as a “valuable and 

distinctive contribution in helping to assure quality of learning” (1995) 
and “a major addition to the Association's quality assurance 
processes. ..(which).!. provides tutors with measures of students' 
engagement in study, the assessment of students' opinions, and the 
evaluation of students’ learning and personal achievements". 
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5. WEAKNESSES AND AREAS OF DIFFICULTY 

• The varying rate of return from learners and the general nature of 
some of their responses reinforced the view that data emanating from 
this approach to learning outcomes should not be used for any 
quantitative purpose. 

® It had proved much more difficult to develop Learning Outcomes on 
the community education programme. Experience here had 
highlighted the following issues: 

o Tutors’ concern about the time required for the process and its 
suitability for some learners 

o The process was more applicable to some subject areas than 
others 

o In Basic Skills/ESOL classes the process was very tutor-led (2). 

® FEFC inspectors reported (1999) that the Learning Outcomes 
approach “does not provide an adequate record of students’ progress 
and the measurable advances they make in learning." 



6. CONCLUSIONS AND NEXT STEPS 

6.1 The reservations expressed by some tutors have led to suggestions that 
for Basic Skills and ESOL students, individual learning goals are more 
appropriate while further work needs to be done to make the Learning 
Outcomes approach more suitable for short, community-based classes 
and their learners. 

6.2 A tutor review/validation of learners’ Learning Outcomes is being 
introduced on an individual student basis in order to address FEFC 
inspector concerns. 

6.3 General conclusions are that: 

* Tutor training and commitment was crucial to the effective use 
of Learning Outcomes 

® The approach was most effective when used together with 
a systematic class visiting programme 

® The Learning Outcomes approach needed to be integrated 
into the course and not bolted on. 
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6.4 Next Steps 



In April 2000 the National Operational Plan contained three new 

objectives in response to the findings of the national seminar: 

® That further advice be given on the management and administration 
of the Learning Outcomes process in order to ensure consistency 
across the WEA 

» That Learning Outcomes for a course should include both those 
specific to the subject and those general to the learning process 

® That the notion of tutor review of students' Learning Outcomes should 
be developed based on 'informed judgement’ (3). 

® That a manageable process for capturing individual learning goals 
appropriate for use on short, community-based courses with 
inexperienced learners, should be developed. 

The timetable for this work would enable full implementation of any 

changes in 2002/03. 



NOTES 

1 . Much of the WEA’s community-based provision is with educationally disadvantaged 
adults who are often taking their first steps back into education since leaving school at 16 
or earlier and for whom formal qualifications at this stage would form a significant barrier 
to their participation. Many learners on the WEA's general education programme, on the 
other hand, already possess a number of qualifications and/or are experienced learners 
who want to help shape the curriculum and determine their own learning objectives. 

2. The WEA is developing a national approach to its Basic Skills and Key Skills work 
whether this is in the form of discrete provision or part of a wider curriculum and whether 
it is qualification bearing or non-qualification bearing provision. All learners on courses 
targeted at educationally disadvantaged adults will undergo an appropriate initial 
assessment to determine their Basic and Key Skills needs 

and will work towards completing successfully an Individual Learning Plan. 

3. The London and Eastern WEA districts have undertaken development work 
in relation to the second and third objectives 
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APPENDIX I - Extract from WEA ‘Notes for Tutors 2001’ 



The next thing you receive will be the Course Outline and Student Report forms. Please ensure that you meet the deadline for 
returning these, as we need time to process them and send you copies for your students. You are asked to inform your Subject 
Officer if for any reason you cannot meet the deadline. 

Please use the form provided or type it out in similar format on your own computer or word processor. You need not type - 
hand-written outlines will be typed up in the office. The outline should be written with your prospective students In mind: use the 
style that you find most helpful to set out what your propose to cover and how you plan to go about it. 

In the course outline, you will need to describe: 

(a) The alms of the course: these need to reflect the active and co-operative nature of teaching and learning within the 
WEA. As such, every class will have as its written general aim the encouragement of active learning within a well- 
structured and supportive environment. Your written aims must work towards this general goaf. 

(b) The main topics you plan to cover, including an indication of the level of the course if appropriate (you do not need to 
specify the content of each session). 

(c) The main teaching learning methods you plan to use. Whatever mixture of methods you plan to use (slide shows, 
group and small group discussions, lectures, practical activities), they must ultimately serve the general unwritten aim 
of encouraging active learning (see above). As with (b), students should be involved in planning and assessments 
through regular discussions as to the appropriateness of course content and method of delivery. 

(d) Any course requirements (preliminary reading, previous study, equipment/materials, etc.) 

(e) Examples of further study or activity to which the course could lead . 

(f) Between threo and six proposed learning outcomes, indicating what you suggest students should gain as a result of 
taking part in the course. 

(g) A short book-list showing essential, background and reference reading . 

Learning Outcomes 

The proposed learning outcomes should provide students with a clear framework for a course. The statement of learning 
outcomes is not a course plan which the class must rigorously follow. Rather the learning outcomes should state the 
knowledge, experience, or other benefits that students might reasonably expect to derive from having taken the course. The 
learning outcomes will identify the skills, forms of understanding, or appreciation of the subject which the course is intended to 
develop. 

When setting the aims and outcomes of the course, try to reflect the active and co-operative nature of teaching and learning in 
the WEA. Students should, as far as possible, be involved in the planning and assessment of their own learning. Teaching 
methods should encourage the exchange and development of ideas and opinions and should be reflected In the aims e.g. 

« enable students to develop the appropriate skills 

« enhance a critical understanding ..... 

• encourage an informed, analytical and creative approach to 

Outcomes should precisely reflect these educational aims e.g. 

As a result of taking part in the course, students should 
® have acquired the appropriate skills to .... 

* have gained knowledge and understanding of the themes/issues/debates within the study of 
® have the confidence to express/discuss ... 

« apply knowledge and understanding gained during the course in their own approach to the subject. 

The booklist asks you to indicate essential, background and reference reading for students. It should not list your sources. A 
paperback covering general aspects of your subject will be invaluable as a course text, if one is in print. 

The course outline with clearly identified proposed learning outcomes is one of the ways in which the WEA can monitor its 
provision and quality of courses, for internal purposes and to meet the requirements of funding bodies. Outlines are read by the 
Subject Officers. Any suggested amendments will be made in consultation with you. When the Outline has been approved your 
will be sent copies for distribution to students and copies are also sent to the Branch. 
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BIRMINGHAM CITY COUNCIL 



Jenny Kirk 



1 CONTEXT 

1 .1 Birmingham City Council’s (BCC) key forum for lifelong learning is the 
Corporate Lifelong Learning Strategy Group (CLLSG). The group has 
representation from Adult Education, Libraries, Sport, Youth, Community 
and Play, Education, Economic Development, Neighbourhood Advice 
and Benefits, Equalities, Policy and Personnel. 

1.2 A research project to look at the range of approaches BCC used to value 
non-accredited learning was undertaken in February and March 2000, 
funded by the DfEE Standards Fund. This chapter focuses on the work 
of four of the Birmingham City Council Departments and Services 
represented on the CLLSG. It describes their adult learning activities 
and the efforts made to recognise achievement. 



2. RECORDING LEARNING OUTCOMES: FOUR EXAMPLES 

Adult Education Service 

2.1 The Adult Education Service is currently developing methods for defining 
non-accredited learning outcomes in beginners’ IT courses, including, 
for example, 'Computers for the T errified’ and 'Make the Most of your 
Computer’. 

2.2 One aim is to offer a “guarantee” of learning outcomes for each student. 

2.3 The exit evaluation questionnaire completed by every student asks for 
information about progression plans. This is being developed by the 
service to elicit details of learners' perception of their learning outcomes. 
Questions asked include: what did you get out of the course? Did you 
get what you wanted out of the course? 

2.4 This process has been initiated in order to address the issue of students 
returning to the same course time after time - indicating that they are not 
recognising their learning and so not moving onto the next stage. This 
causes a bottle neck for new participants who want to learn IT - a major 
need amongst non participants - and so has implications for preventing 
Widening Participation. 
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Economic Development Department 

2.5 Economic Development has a lead role in economic regeneration 
with a focus on community capacity building and business workforce 
development - particularly small and medium sized enterprises (SMEs). 
It also supports a network of 25 Employment Resource Centres (ERCs). 
These provide vocational advice, guidance, information and support to 
unemployed people. Many of the ERCs are not directly involved in 
formal training; others have partnerships with colleges or other training 
providers, who provide tutors for the in-house delivery of accredited 
training. 

2.6 Examples of non-accredited learning taking place at ERCs include: 

® Open and flexible learning in core skills using IT and print-based 
resources* 

® Drop-in access to computers for internet and general IT familiarity, 
informally supported by ERC staff.* 

© Group and individual work on how to produce CVs, together with job 
search, interview skills etc. 

• Informal training and advice to clients considering setting up 
enterprises, prior to referring them to enterprise agencies. 

® Information sessions on how the education and health systems 
operate, particularly for minority ethnic communities. 

® Confidence building and assertiveness sessions. 

© Customised training for ERC management committees (usually local 
residents acting in a voluntary capacity).* 

2.7 Participants on some courses (marked *) complete an evaluation form. 

2.8 There is a registration procedure and some tracking of learners' 
progress. 

2.9 Quarterly monitoring data is collated annually. 



Action Learning for Community Enterprise and Development (ALCED) 

2.10 The Action Learning for Community Enterprise and Development 
(ALCED) is a 2-year ESF (European Social Fund) project which has 
local and trans-national partners. The aim of the project is to provide 
an action learning opportunity for up to 150 adults from the inner city 
areas of Birmingham (the unemployed, lone parents, Black and other 
minority ethnic people). The project is designed to empower groups 
and individuals to translate their own vision for community enterprise 
into concrete practical proposals and business plans, as well as 
develop their capacity for community leadership and participation. 
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2.11 At the start of each course a Skills Audit is undertaken in each group to 
identify the skills and knowledge which individuals already have, those 
they want to improve and those they want from their mentors. 

2.12 Review and evaluation is built into each learning programme - midway 
and upon completion. 

2.13 Each participant is given the opportunity to develop an exit strategy from 
the programme, in a residential study context, with one-to-one support 
available. 

2.14 All participants are encouraged to run a project, or look into an issue of 
concern to them. This provides the vehicle for making learning relevant 
to the needs of each person. 

2.15 An essential learning outcome is also the production and presentation to 
a group of people of a computer generated presentation about 
individuals, or group learning experiences^ This takes place in 
workshops, seminars and conferences in the UK and in European cities. 

2.16 Every participant is encouraged to produce a testimony. They reflect 
upon their experience of action learning, highlight key learning points and 
identify improvements they would like to see built into the programmes 
together with their next steps in terms of action planning. 

2.17 A variety of certificates such as Action Learning Certificates and 
Certificates of Achievement are produced and presented to learners 
at a Celebrating Success Awards Ceremony. 



Education Department Schools Advisory Service 

2.18 As a result of a variety of initiatives, schools are being recognised as a 
key community outlet for delivering learning to adults in partnership with 
Further Education Colleges and the Adult Education service. This is 
described as Family Learning or Family Education. 

2.19 Through a range of carefully planned activities and projects, Family 
Learning activities seek to get one or more people from a family group 
involved in learning which they can then share and develop with other 
members of their family. 
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2.20 Two examples of on-going projects: 

(a) Two hundred of the 350 Primary Schools in Birmingham (and some secondary 
schools) offer an annual workshop session for parents of every child on the register. 
The focus is on the child - about parents helping their children to learn. Success of the 
service is measured in terms of attendance (90% of parents take up the invitation). 
Sample surveys are undertaken to evaluate the impact of the service. 

(lb) One Standards Fund project has involved the delivery of four taster programmes of 
Family Literacy or Family Numeracy each comprising three workshops (a total of 6 -9 
hours class contact). Forty adults have benefited from the project and the aim is to 
deliver six programmes next year. Although some participants may begin to work 
towards a unit of Wordpower or Numberpower, there is as yet no customised 
accreditation. 



2.21 The Basic Skills Agency Kit provides checklists and guidance for tutors 
to engage the adult learners in a self-assessment process. However, 
some Basic Skills’ tutors have found that the steps of achievement are 
too big and the Kit therefore is inappropriate for their learners (there may 
be a staff development issue here). 

2.22 Work is currently being undertaken to encourage the Adult Education 
tutors to identify skills and knowledge derived from the National 
Curriculum. 



3. BACKGROUND 

3.1 The first stage of the project involved working with Steering Group 
members in order to begin to map non-accredited adult learning activities 
provided or funded by BCC, share experience of recognising learners' 
achievements in non-accredited learning and consider ways of assessing 
the value of non accredited learning activities. 

3.2 A survey was carried out in order to gather information on the range 
of adult learning activities delivered by departments or sections of the 
BCC. Subsequently, semi-structured interviews were undertaken by the 
author in a series of one-to-one meetings with representatives from Adult 
Education, The Library Service, Equalities, Economic Development, 
Schools Advisory Service and the Neighbourhood Advice and Benefits 
sections. 



4. STRENGTHS 

4.1 There is a range of excellent practice in collecting learners’ own 
perceptions of achievement, student satisfaction data and course 
evaluation comments 

4.2 Defining learning outcomes for all learning programmes and recording 
small steps of achievement is perceived to be an essential basis for 
learning by Steering Group members. 
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4.3 The work was found to complement the quality improvement strategies 
across City Council Departments where there is no systematic and 
common means of assessment. 

4.4 The work was perceived as a project which would serve to inform a wider 
range of partners, for example community and voluntary groups who are 
involved in the delivery of adult learning opportunities. 



5. WEAKNESSES AND AREAS OF DIFFICULTY 

5.1 There is no common approach or instrument of measurement across 
Departments. It is therefore difficult to provide information in a consistent 
and systematic way to funders, politicians and other stakeholders 
including partners. 

5.2 Though Open College Network and other forms of accreditation are 
widely used to provide learners with formal recognition of their 
achievements, there is evidence that some learners who want it do not 
have the opportunity of submitting their work for accreditation. 

5.3 There is a range of issues concerning the role of the 
tutor/facilitator/trainer/learner supporter and their professional 
development needs. For example: how is quality of delivery to be 
assured? 



6. NEXT STEPS 

6.1 A number of recommendations for action were agreed by the project 
Steering Group. 

6.2 A corporate learning outcomes strategy would be developed to enable 
Departments to assess (both quantitatively and qualitatively) the impact 
of non-accredited adult learning in terms of economic outcomes, social 
participation and community benefits and personal benefits. 

6.3 All Steering Group members would consider the implications of the 
research undertaken for the project and take into account when 
developing their individual Department's plans. 

6.4 A body of evidence would be gathered which makes the case for the 
value of small steps of learning achievement. 
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4. RECOGNISING AND VALIDATING NGN- 
ACCREDITED ACHIEVEMENT: THE LEARNER’S 

PERSPECTIVE 

An interim report on the N8ACE research 
Cheryl Turner and Kate Watters 



1. Introduction 

1 .1 Learners are both subjects of and participants in the different 
approaches to identifying achievement in non-accredited or ‘progression’ 
learning that are outlined in this publication. But what are their views of 
this endeavour? How do they weigh the benefits and disadvantages of 
their engagement and what impact does this have on their experience of 
learning? 

1.2 A deeper understanding of learners’ responses to assessment processes 
is crucial to the realisation of the Government’s vision of a demand-led 
planning system and to the achievement of its ambition to raise 
standards and levels of participation, particularly in more informal, 
community based provision. 

1 .3 From the inception of this research, the intention was to complement 
the work of the Learning and Skills Development Agency (LSDA) 

in identifying generic guidelines for effective practice (supported by 
illustrative case studies) with a parallel enquiry by NIACE into the value 
of such practices to learners. This reflects the first theme identified in the 
Introduction - the centrality of learners - and the first of the unresolved 
questions listed in the final ‘Overview’, “How do learners benefit from 
the process of recording and validating achievement?” 



2. Context 

2.1 The Remit Letter from the Secretary of State to the Learning and Skills 
Council (LSC), setting out the operational priorities for the Council’s first 
year, makes it clear that the LSC is expected to have a strong focus 
upon raising standards. 

It [the LSC] will need to introduce robust arrangements to tackle deep 
seated problems of inconsistency of standards by challenging poor and 
coasting providers, and by ensuring excellence in teaching and training 
which meets the needs of all learners, and which promotes equality of 
opportunity 1 
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2.2 All providers funded through the LSC will have a responsibility to 
establish and sustain a culture of continuous quality improvement. 

The main elements of this process will be self-assessment and 
development planning (a key contractual requirement), monitoring 
and review, and external inspection. 

2.3 The documentation argues unequivocally and consistently that this is 

meant to be a learner-focused system. For example, the guidelines on 
Self Assessment and Development Plans maintain that “The new 
arrangements have been designed to ensure that the interests of the 
learner come first and are of paramount importance’’ and that the 
underpinning principles are there to achieve the “objective of placing the 
learner at the heart of the system”. 2 

2.4 The LSC, as part of its Corporate Plan, will publish a 
Qualitylmprovement Strategy, which will shape and drive this forward. 
This will be informed by the outcomes of various consultation processes 
including the enquiry into raising standards in post-16 learning, proposals 
for developing practitioner skills and qualifications in FE, work-based 
learning and adult and community learning, and finally, the Common 
Inspection Framework 3 . 

2.5 The Common Inspection Framework (CIF) will guide the 
independentinspectorate OFSTED and the new Adult Learning 
Inspectorate (ALI) in their review of all LSC/Employment Service (ES) 
funded provision, including non-accredited programmes. As might be 
anticipated, it also accords centrality to learners. We are told that 
"Inspections will focus primarily on the experiences and expectations of 
individual learners” and will set out to answer the fundamental question, 
“How effective and efficient is the provision of education and training in 
meeting the needs of learners, and why?” 4 . 

2.6 The Framework is built around seven broad questions, each of which is 
defined by a series of evaluation statements. Among these, inspectors 
are required to address the following: 

• learners’ progress relative to their prior attainment and potential 

• the development of personal and learning skills 

® how well teaching and training meet individuals' needs and how well 
learners learn and make progress 

• the uses of assessment in planning learning and monitoring learners’ 
progress 

• the extent to which courses or programmes match learners’ 
aspirations and potential, building on prior attainment and experience 

• the diagnosis of, and provision for, individual learning needs 

• how effectively, through quality assurance and self-assessment, 
performance is monitored and evaluated and steps taken to secure 
improvement 
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9 how well equality of opportunity is promoted and discrimination 
tackled so that all learners achieve their potential 

2.7 The italics and emboldening have been added to emphasise the 
significance within the Framework of the careful assessment and 
recording of the progress of individual learners against specified 
(and challenging) learning objectives taking account of their earlier 
attainment, potential and aspirations. 

2.8 It is clear that learners’ views will be a key source of evidence in this 
context. This raises questions however about the extent to which their 
comments will require verification, the ways in which 'distance travelled' 
will be gauged, and the impact of the whole process upon the learning 
experience. 

2.9 For example, initial assessment could provide some of the necessary 
data for identifying 'distance travelled’. However, while it is an important 
element of good practice for some areas of learning, such as basic skills, 
initial assessment would be inappropriate for some types of provision 
and some groups of learners. This could include many of the 
characteristically shorter courses in non-accredited programmes, and 
without skilful deployment, more hesitant learners. 

2.10 Inevitably, the monitoring and assessment process comes under close 
scrutiny. Inspectors will be expected to consider the suitability and rigour 
of assessment and recording methods, whether these are fair, accurate 
and carried out regularly, and the accuracy of the information produced 
for programme planning. Assessment is seen as both formative - an 
integral part of the learning process - and summative. The views of 
learners on these processes will be crucial, particularly in relation to the 
issues of suitability, fairness and accuracy 

2.11 While inspection against the CIF is expected to take place on a four year 
cycle the responsibility for ongoing monitoring and quality assurance, 
including for non-accredited programmes, sits with either the local arms 
of the Learning and Skills Council (LLSC) or the ES. Their work is 
intended to complement that of the inspectorate and to operate within 
the same frame of reference. 

2.12 Providers, the LLSCs ands the ES are charged with working together 
to ensure continuous improvement. Provider self-assessment reports, 
which will be one of the key instruments, are expected to address the 
quality statements in the CIF and to refer to the list of evaluative 
statements when grading the quality of their provision. This will apply 
to all provision regardless of the number of learners, and to all learners 
irrespective of, for example, their age, ability, gender, ethnicity, learning 
difficulty or disability. 





3. NIACE Research - The Voice of the Learner 

3.1 Designing and articulating systems privileging the needs, aspirations and 
achievements of learners is one thing. Creating opportunities for learners 
to reflect on those systems and to influence them, is another. This study 
aims to give learners a voice in the discourse and to reflect the overall 
intention of the Government’s policy. 

3.2 It seeks to address the philosophical as well as practical challenges 
involved in working with learners to obtain assessment processes which 
reflect their requirements and aspirations and which offer robust 
evidence of achievement and quality improvement for the purposes of 
external scrutiny and funding. 

3.3 This is underpinned by the belief that making spaces for learners 

to be heard in any evaluation of the systems created to assess their 
achievement is vital, not just to the effective implementation of those 
systems (or to the design of better ones), but also to the empowerment 
of learners. 

3.4 This is the key to the creation of a learning society. The final report of 
the Policy Action Team on Skills argued that one of the reasons for 
the failure of current arrangements to meet the needs of many people, 
particularly those in socially disadvantaged areas, is the absence of their 
involvement in and ownership of the learning process 5 . This reflects 
McGivney’s view that, 

Research has shown that disadvantaged groups in the community do not embark on 
learning pathways as a result of exercising choice over what is already on offer but as a 
result of organisations responding to identified interests, negotiating options and 
customising provision 6 

3.5 Taking account of learners’ views is a tenet of good practice in the 
identification of the objectives of the learning process. It applies equally 
to the ways in which that process is interrogated in order to identify and 
validate its outcomes. 

3.6 The NIACE research has the following objectives: 

1 . To identify learners’ perceptions of the value and effectiveness of 
different approaches to validating the gains from non-accredited 
learning 

2. To identify learners’ requirements of and judgements about current 
approaches to validating the gains from non-accredited learning 

3. To make an evidence-based contribution to the ALI and LSC 
understanding of 

quality assurance in relation to non-accredited learning 
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4. To contribute a paper on possible criteria and measures of learning 
gain in non-accredited learning contexts for consideration by the 
Department for Education and Employment and LSC officers in 
developing funding and quality assurance models 7 

4. Scope 

4.1 The primary focus of the activity is on action research involving learners 
and those responsible for planning and managing the learning 
opportunities they enjoy. This addresses the following questions: 

® What do learners think and feel about being involved in learning 
outcome and assessment systems? What are the perceived benefits 
and disincentives? 

® Are their requirements of such systems the same as other stake 
holders e.g. teachers, managers, inspectors, funders, policy makers 
and implementers? 

® What are learners’ opinions of the effectiveness of the learning 
outcomes and assessment systems they are involved with? Or of 
alternative systems? 

• What do learners think and feel is the impact of their involvement in 
such systems on their relationships with 

- other learners 

- teachers 

- the curriculum 

- teaching and learning methods/approaches 

» How and what can learners’ perceptions about such systems 
contribute to the development and implementation of quality 
assurance arrangements and frameworks; to staff development; and 
to the provision of support for learners? 

4.2 The action research is supported by desk research that seeks to 
ascertain the following: 

• What approaches to learning outcome and achievement systems are 
currently experienced by adult learners? 

» How are they similar or different? What common features do the 
various systems have, as perceived by learners and those who work 
with them? 

® What issues for learners and those who work with them are 
associated with implementing the various approaches? 

• What types of judgements and what range of judgements are 
learners being asked to make about their learning progress/gain? 
How are they being asked to do this? 

4.3 The underpinning intention is to link the outcomes of this enquiry to the 
application of the CIF and the LSC's understanding of appropriate and 
effective ways of assessing learning gain. More particularly, it is hoped 
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to inform their interpretation of how learners’ views can contribute to both 
the discussion and development of practice at local and national levels. 

4.4 The outcomes could also contribute to the development of support 
measures for quality assurance in the adult and community learning 
sector. This includes staff development and training, mentoring, 
quality improvement networks and subsequent good practice guides. 

5. Timescale 

5.1 The secondary and primary action research phases are now underway 
and the research activity will continue into the early summer with a view 
to completion in July. The findings will be included in an Autumn 2001 
publication from NIACE. 

6. Methodology 

6.1 A key requirement for this initiative has been to devise a methodology 
that is appropriate and non-threatening to learners. This includes 
learners “who have had few positive experiences of education in the 
past and/or have minimal language and literacy skills” 8 At the same 
time the methodology has to be sufficiently rigorous to give credence 
to the outcomes of the research process. The approach has to be 

'fit for purpose’ and relevant to local situations. 

6.2 Inevitably, some of the research has been carried out within or around 
the learning situation as most learners on non-accredited programmes 
study part-time. This affects the pace and method of enquiry as learning 
opportunities are precious and not to be interrupted lightly or beyond the 
tolerance of the individuals involved. 

6.3 In practice, learners’ views and perceptions are being captured and 
recorded in a variety of situations. The determining factors are: the 
availability of physical space adjacent to the learning activity; the 
willingness of individuals or groups of learners to accommodate an 
interviewer within their learning time; the opportunities for meeting with 
learners individually or in groups at times outside their learning time; 
and above all, the readiness of learners to participate in the research 
activity. Learners are consulted prior to arrangements being made. 

6.4 The principal methods are semi-structured interviews and focus groups, 
depending on the practical considerations indicated. A series of open 
questions prompts learners to reflect on: 

® their expectations about learning at the outset 
» their experiences of working with their tutors to identify and record 
appropriate learning outcomes 
® how their learning progress has been assessed through 
self-assessment and/or formative assessment by the tutor 
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® their recognition of learning gain on completion of the programme 
through review or some form of summative assessment. 

» the totality of gains from their learning experience which may 
extend beyond the planned learning outcomes 
o how the particular system of evaluating learning gain impacts 
upon their learning 
® any benefits and disadvantages 
® suggestions for improvements. 

6.5 Two instruments have been developed to facilitate discussion with 
learners. In order to explore the way in which learners and tutors can 
work together to identify learning outcomes and evaluate learning gain, a 
diagram has been prepared which suggests how this might happen, 
looking at the role and responsibilities of the learner and the tutor, and 
the nature of the process in practice. 

6.6 Second, a typography of potential ways of demonstrating and assessing 
learning gain has been developed. This includes individual and group 
activities which could provide ways of generating evidence of skills 
acquired or extended, knowledge 'owned' and used in context, 
understanding of concepts or the application of theory, all as a by- 
product of purposeful and hopefully enjoyable learning experiences 
integrated within a learning programme. Learners are being asked to 
comment on the relevance of these activities to their programme and to 
their recognition of learning gain. 

7. Sample 

7.1 The main target group comprises learners who are or have been 
accessing programmes with a structured approach to the identification 
of learning outcomes and the assessment of learning gain. The case 
studies presented in the rest of this publication are an important source, 
along with a range of other providers from the adult and community 
learning sector. A range of other providers from the Further Education 
and Local Authority sectors offers further sources. 

7.2 This research is founded on an understanding that learners do not 
present a homogeneous set of needs, aspirations or intentions. Their 
very diversity, and the challenge this poses to assumptions behind 
current systems and planning, is part of the rationale. Therefore the aim 
is to consult as broad a range of learners as possible taking account of 
age, ability, gender, ethnicity, learning and support need, and 
programme range. 

7.3 This involves accessing a variety of non-accredited courses and 
programmes, including (but not limited ) to the liberal and performing 
arts, humanities, languages, skills-based programmes of various kinds 
and those concerned with personal development. Enquires are also 
underway to include FEFC Non-Schedule 2 pilot projects and 
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community-based initiatives funded through the Adult and Community 
Learning Fund. 



8. Links with quality assurance 

8.1 At a later stage, the action research will include semi-structured 
interviews with tutors and managers who operate learning outcomes and 
assessment systems. The intention is to identify the relationship between 
the application of these systems in terms of their value to learners, and 
the outcomes of their application in terms of quality assurance. 

8.2 This will include exploring the way in which the development of such 
approaches has informed the growth of wider quality assurance systems. 
Most importantly the research will look at how the involvement of 
learners in identifying learning outcomes and learning gain has informed 
curriculum and staff development, thus contributing to a virtuous cycle of 
continuous improvement. 



9. Links with other initiatives 

9.1 The design of the NIACE research project and the scope of the activity 
have been informed by a number of other initiatives. These include the 
lessons emerging from the evaluation of the work supported by the 
FEFC Non-Schedule 2 projects and the Adult and Community Learning 
Fund projects. It will also take account of a parallel LSDA project 
focusing on learning outcomes and assessment approaches in the 
context of ESOL and Basic Skills 9 



10. Emerging issues 

10.1 The action research will be completed by early May and its findings 
presented in a report available in early summer and a publication in late 
autumn. Although it is too early at this stage to discuss the outcomes 

it is possible to comment upon a number of themes identified within 
the 'Introduction 1 and ‘Overview’. 

10.2 It is important to begin by welcoming the strong consensus on the 
centrality of learners to the processes described in the publication, and 
the acknowledgement that this remains one of the major lacunae in what 
is already a very substantial body of knowledge about effective practice. 

10.3 Clearly the weight of argument in this publication, as elsewhere, requires 
that our judgements of whether assessment systems are ‘fit for purpose’ 
should be informed by learners' perceptions of what that means. There is 
tension, however, in the fact that learners' purposes are likely to be many 
and various and may differ within a learning group. Furthermore, they 
may not accord with those of the tutor or managers concerned with 
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quality assurance - or indeed external views as to the purpose of 
learning and its evaluation. 

10.4 As was stated in the 'Introduction’, clarity about the purpose of 
assessment is essential. This will inform discussion about both the role 
and status of learners self-assessment in any record of achievement and 
about the necessity for validation of their views. As part of this, we need 
to ask what learners think about validation of their learning and about its 
relevance to their purposes. 

Appropriate levells of involvement 

10.5 We also require a better understanding of the extent to which learners 
seek involvement in or information about other aspects of the process. It 
might be ethically or methodologically desirable to ensure that they are 
aware of all aspects of the assessment process, but do learners want or 
require detailed knowledge of the way in which they contribute to the 
providers’ quality assurance systems? 

10.6 For example, learners are unlikely to know about, or perhaps be 
interested in, efforts to find an appropriate way of deriving internal 
benchmarks from aggregated information even though the genesis of 
this is their individual self-assessment. Of greater immediate relevance 
will be their own, individual perception of how they are getting on, what 
they feel about what they are doing, and what they have learned. 

Identifying and demonstrating ‘soft’ and group outcomes 

10.7 Perhaps of equal importance to this sense of individual achievement will 
be a recognition, via assessment, of the social and interactive aspects of 
learning that many learners prize. This moves us into the more 
problematic realms of interpreting collective gain and ‘soft outcomes’ 
such as self-esteem and greater confidence. As is noted in the 
’Overview’, while such outcomes are of immense importance, and 
perhaps of greatest value to many learners, they are hard to define and 
their acquisition is hard to verify. 

10.8 Qualitative rather than quantitative material would be used to 
demonstrate this type of achievement and gain from learning. These 
sources offer the most direct and unfiltered evidence of learners’ 
perceptions and therefore, while they are particularly revealing, they are 
also the most subjective. Again, as is noted in the ‘Overview’, the place 
of qualitative evidence in the record of assessment remains an important 
outstanding question. Learners’ views are crucial to any resolution. 

Obtaining a common language 

10.9 All these questions are affected by the overarching issue of knowledge 
and what learners need to know about the interpretation of learning 
gain in order to reflect their views fully and accurately. For example, 
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the ‘Overview’ contends that we need to know more about the nature 
and value of unanticipated outcomes and the wider benefits of 
learning 10 .While endorsing this, it is arguable that we need to move 
our thinking back a stage to enquire into the meaning of such concepts 
for learners. In other words, are current systems and instruments 
predicated on an understanding of the learning process that most 
learners, and particularly newer ones, are unlikely to share? 

10.10 Furthermore, we should consider whether or not those concepts are 
communicated in ways that are congenial and understandable to 
learners and therefore likely to facilitate more reflexive responses. 
Consultation with learners is an essential part of constructing and 
sharing a common body of concepts and terms that can be used to 
analyse the learning process and which will produce more revealing 
information about what it achieves. This is crucial to the development 
of the kind of evidence-based planning processes that the Government 
wants to see informing the work of the LSC. 

Interpreting ‘distance travelled’ 

10.11 Once we step outside the qualifications and credit framework, many 
of us become unsure of the ways in which learning gain can or should 
be measured and verified in order to create an estimation of ‘distance 
travelled' or ‘value added’. This publication, along with others, 11 makes 
reference in this context to the role of initial assessment, appropriate 
benchmarks, and the interpretation of ‘soft outcomes’, amongst other 
issues. 

10.12 This is of crucial importance to the development of systems and 
practices that will impact directly upon the quality of learners' 
experiences. Again, proper consultation with learners in these areas 
depends not just upon commitment to the principle but also upon the 
creation of a common language and frame of reference. This requires 
a shared sense of the application of certain widely used terms - such 
as 'learning outcome’, 'validate' and ‘measure' - in order to avoid a 
Humpty Dumptyish confusion arising from particular preferences 

for the meaning of words. 

Working with diversity 

10.13 Finally, it is worth emphasising that the sheer diversity of learners and 
learning programmes presents a major challenge to the development 
of systems intended to meet learners’ needs and to reflect their 
responses. It cannot be assumed that men and women with unique life- 
experiences and from differing cultural backgrounds will respond in the 
same way or with equal confidence to self-assessment processes 

and questions. Neither can it be assumed that learners with differing 
purposes and who access learning opportunities which belong within 
different academic or vocational traditions, or indeed break with 
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‘tradition’ and seek to innovate, will respond to such processes 
and questions uniformly. 

10.14 Learners in the creative arts are notoriously unlikely ever to claim that 
they "can do” something really well; it is the journey towards 
accomplishment that is often what really matters. Similarly, what kind 
of scientist will claim ever to fully know or understand phenomena? 
Many of the learners now participating in learning experience difficulty 
in claiming that they are learning and making progress, and would 
hesitate to describe themselves as ‘competent’ in any context. 

10.15 The case studies in this publication offer eloquent testimony to the 
need to ensure that the introduction of any systems for recognising 
and validating non-accredited achievement is inclusive of the views 
of learners. The main task for this study is to build on that principle 
and the insights offered in this and other sources, to create a body 
of information about learners’ perceptions that will inform both policy 
and practice in this area. 
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5. RECOGNISING AND VALIDATING NOW-ACCREDITED 

ACHIEVEMENT: 

AN OVERVIEW 1 



1. INTRODUCTION 

1 .1 It is time to take stock: to draw out the lessons from the case studies, 
and provide an overview of what we know, what we don't know, and 
what remains to be done. 

1 .2 The chapter is arranged as follows: 

* A Framework for Good Practice 

• Areas for Development 
® What We Don’t Know 

® Conclusions and Emerging Issues 

1.3 We begin by setting out what we can now regard as common knowledge, 
and we describe the nine stages that make up a framework for good 
practice. We then move on to discuss the implications for staff 
development, managing change and quality assurance, before itemising 
lacunae in our knowledge, the importance of which is not to be under- 
estimated. Finally, we identify emerging issues that await further 
research. At regular intervals we refer to the Common Inspection 
Framework' 2 pointing up connections between the work described 

here and the requirements set out in this inspection document. 

1 .4 There is general agreement about some of the most important 
ingredients of good practice - indeed, they have been known about for 
many years. What these latest case studies bear out is that a significant 
part of the process of recording and validating learning is now crystal 
clear and beyond dispute. The point of greatest importance, and on 
which there is consensus, is the centrality of learners to the process 
they are a part of. The success of all attempts to validate learning 
depends on our efforts to listen to learners, to understand and appreciate 
what they want and need, and then to devise a system that is congenial 
to their needs and supportive of their aspirations. 

1 .5 We must emphasise that much of what follows is not new. 3 Amongst 
others, NIACE 4 , the WEA 5, and FEDA 6 (now the Agency) have all 
produced materials which in many particulars correspond closely to 
what follows. It is this common ground - the basic ingredients of 
good practice, areas for development and the remaining gaps in 
our knowledge - that we endeavour to set out in what follows. 





2. A FRAMEWORK FOR GOOD PRACTICE 



The Bottom Line 

2.1 If we look at what the case studies are telling us, and if we look also at 
evidence from other providers, we can safely say that what recording 
and validating adult achievement boils down to is this: 



An assessment of learner’s progress as judged against 
their learning objectives 



Nine Stages 

2.2 If we break down this bottom line, we uncover a process that includes 
nine stages: 



Stage 1 : 
Stage 2: 
Stage 3: 

Stage 4: 
Stage 5: 
Stage 6: 
Stage 7: 
Stage 8: 
Stage 9: 



initial identification of learning objectives 

initial assessment of learners 

negotiation of learning objectives 
with learners 

learner self assessment 

on-going formative assessment 

progression advice and guidance 

record of achievement 

moderation of assessment 

celebration of achievement 



2.3 What you see above is the basic framework for recording and validating 
adult learning, and the case studies collected here, in common with the 
recorded experiences of other providers, exemplify to a greater or lesser 
degree the nine-staged process. 

About The Framework 

2.4 These nine stages constitute the basic framework which institutions 

can adapt or make use of in ways that best suit their own circumstances. 
They are not intended as a rigid blue-print which makes no allowance 
for individual circumstances. 
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2.5 As you will be aware from reading the case studies, not all steps apply 
to all courses and all learners in the same way. In some cases involving 
students with learning difficulties or disabilities, steps 3 and 4 may not 
be appropriate at all, whilst step 2 will be necessary in many such 
cases, but undesirable in many others. 

2.6 In all cases without exception tutors are required both to identify learning 
objectives and to assess learners against these objectives. Assessment 
is integral to the entire process. If the tutor does not understand this, and 
if it is not a central feature of teaching and learning, the process will fail 
to benefit anyone. This lesson is underlined again and again in the 
experience reported here. 

2.7 However an institution chooses to make use of the framework, and 
whichever stages it chooses to focus on, the resulting method must find 
support amongst learners, and prove beneficial to their learning. But in 
order to acquire the necessary support in the first place, it is vital to listen 
to what learners have to say before reaching decisions about a preferred 
method for validating learning. 

2.8 What follows is not intended as a comprehensive guide to good practice; 
for this there are plenty of admirable publications already in circulation. It 
includes only a sketch of each of the seven stages, together with a few 
words about matters to watch out for, as highlighted in the case studies 
in this volume. 

Stage 1: Initial Identification of Learning Objectives 

2.9 Tutors should clearly and accurately identify the primary learning aims 
and objectives for learners in advance of the course. The aims are 
understood as the general or overarching aspirations for all learners; the 
objectives are those more particular goals that the tutor will seek to 
confirm (and amend if necessary) with the learners. 

2.10 With respect to all the agreed objectives, it should be possible to 
estimate the amount of progress learners have made. 

2.11 Inspection will consider the extent to which teachers and trainers 'plan 
effectively with clear objectives that all learners understand . 7 Step 1 is 
the first (but not the only) opportunity for the tutor to identify what he or 
she thinks these should be. 

Stage 2: Initial Assessment 

2.12 Initial assessment is necessary for some groups of learners, for 
example some students with learning difficulties and disabilities. It may 
be unnecessary and even positively undesirable for other groups of 
learners, for example some marginalised and hesitant learners for whom 
the prospect of formal assessment will act as a deterrent. (Assessment 
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may also be informal and carried out in a manner that is non- 
threatening.) 

2.13 Initial assessment serves several purposes, aiming amongst other things 
to provide an opportunity for: 

© gathering information about learners’ knowledge, experience, and 
preferred learning styles 
® assessing additional learning support needs 
« discussing and negotiating the learning programme 
© adapting the programme to meet learners' needs 

2.14 Inspection will look at the extent to which ‘initial assessment provides an 
accurate basis on which to plan an appropriate programme of work' s 

Stage 3: Negotiation of Learning Objectives with Learners 

2.15 At the start of the course the tutor should be able to share with learners 
the aims and objectives as these appear in the scheme of work. 

2.16 The aims and objectives should be discussed so as to identify both those 
suited to the class as a whole, and those suited to the needs and goals 
of individual learners. 

2.17 Not all learning objectives need apply to all students. Some learners may 
have learning objectives of their own, and indeed should have, if these 
are conducive to their own learning, and are compatible with the overall 
aims of the course. It is quite possible to be responsive to differential 
learning needs and preferred learning styles without abandoning the 
central aims and objectives of the course. 9 

2.18 Any new objectives should be discussed and agreed with students. A 
written record of the revised list of objectives should then be prepared, to 
which both tutor and learners are able to refer. 

2.19 Inspection will evaluate the extent to which tutors ‘with learners, develop 
individual learning plans, informed by initial assessment, that are 
reviewed and updated regularly" ' 0 

Stage 4: Learner Self Assessment 

2.20 Learners should be encouraged to become aware of what they are 
doing, how they are doing it, and what is going well or badly. On-going 
dialogue amongst learners, and between tutor and learner, can be 
structured so as to enable learners critically to reflect on their learning 
and their progress. 
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2.21 It is now recognised that ‘learning to learn’ is an important feature of 
the learning process, in which the recognition and evaluation of one’s 
own progress plays an important part. Facilitating self assessment 
contributes to the process of taking learner perspectives seriously, 
encouraging learners to develop their knowledge of what they are 
doing, and the learning styles that work best for them. 

2.22 Inspection will examine the extent to which learners 'develop the skills of 
critical evaluation’, 11 and ' understand . . . how well they are progressing 
and what they need to do to improve' 12 

Stage 5: On-going Formative Assessment 

2.23 Formative assessment is integral to the process of recording 
achievement, and requires of tutors that they constantly attend to the 
work of the learners, noticing any progress or lack of it, encouraging 
and listening to their reflections, and providing appropriate feedback in 
response. This is essential to the entire process of validating learning, 
and is in any case simply good teaching practice. 

2.24 Many tutors conceive ’assessment’ as formal, written and summative. 

But it need be none of these things. On the contrary it will often be 
informal, oral and formative, and should always be designed to be 
encouraging and supportive. ‘Assessment’ in this context implies no 
more (and no less) than a discerning judgement about learner’s 
progress; it is ‘on-going’ in the sense that it goes on all the time; and it 

is formative in so far as its purpose is forward-looking, aiming to improve 
future learning (as distinct from the retrospective nature of summative 
assessment). 

2.25 We should not conclude from the fact that students have opted not to 
pursue accreditation that they therefore want nothing to do with 
assessment. This doesn’t follow and it isn’t true. Learners do not want to 
be either ignored or dismissed, but nor do they want to receive endless 
uncritical compliments. What they do tend to want is insightful, sensitive 
and detailed feedback from a tutor who is both supportive and continually 
aware of what is going well and not going well. 

2.26 Inspection will look at the extent to which 'assessment is used to monitor 
progress and inform individual learners about how they are performing 
and how they might develop further ’ 13 

Stage 6: Progression Advice and Guidance 

2.27 What was previously referred to as ’non-accredited provision’ or 
‘non qualification bearing provision’ is now to be referred to as 
‘progression provision’. 
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2.28 It is a recognised feature of good practice to provide advice and 
guidance to students about further courses, learning programmes, 
employment and other opportunities that are related to the course they 
have completed. 

2.29 Inspection will look at the extent to which ‘careers education and 
guidance are effective in guiding learners towards the opportunities 
available to them when they have completed their studies or training ,M 

Step 7: Record of Achievement 

2.30 To repeat: a record of achievement is in essence an assessment of 
learners’ progress as judged against their learning aims and objectives. 

2.31 The record should refer to clearly identified learning objectives. This may 
seem obvious, but some records fail to mention the objectives and 
instead are taken up with only general, ill-focused remarks bearing little 
relation to learning. 

2.32 The record should take account of classroom observation, learner self 
assessment and tutor assessment. 

2.33 The record will always include quantitative evidence, which is to say 
statistical information about each learner, and will refer to attendance, 
retention, and - wherever possible - what is known about progression 
beyond the course. 

2.34 It is almost always desirable to supplement quantitative evidence with 
qualitative evidence; this may include a portfolio of work, a performance, 
a piece of writing, and so on. 

2.35 The following is an example of what a record of achievement might 
include, though the record required will depend on the nature and length 
of the programme (with lesser demands made of very short 
programmes): 

• syllabus (outlining rationale, content, aims, and delivery methods); 

• learning objectives for each learner; 

® recorded outcomes along with programme evaluations; 

• note of progression opportunities and any learner progression; 

• record of assessment and summary of the programme 15 

2.36 There is a great deal more to be said about what a record of assessment 
should consist of, but since much is less than certain we reserve further 
remarks for section 4.3 below. 

2.37 Inspection will look at the extent to which 'achievements towards learning 
goals are recorded’ 16 and also at the extent to which ‘assessment 
information is used to guide course and programme development'' 7 
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Stage 8: Moderation of Assessment 

2.38 Internal moderation enables validation of tutor assessments. Tutors 
make presentations of samples of assessed work to colleagues together 
with a Head of Department or Curriculum Manager. The moderating 
process will include close scrutiny of presentations and 
recommendations for amendments. 

2.39 The purpose of internal moderation is: 

» to share and disseminate good practice 

• resolve issues in difficult cases 

e identify common standards and criteria for assessment 

2.40 Once a tutor's assessment is judged as reliable, it may be possible to 
take some of their future judgements on trust, in the sense of requiring 
less { not ’no’) evidential support than previously. This both takes account 
of the need to validate assessor and assessment, whilst also recognising 
the need to limit demands for evidence to whatever is proportionate to 
the institution’s resources 18 

Stage 9: Celebration of Achievement 

2.41 It is widely agreed that the learner should be at the heart of the learning 
outcomes process, and this is indeed the first principle of the Learning 
Outcomes Group. 19 Nothing is more important for the endorsement of 
their learning than an event at which their success is celebrated. 

2.42 It is not a sentimental but a plain truth, for which experience provides 
adequate evidence, that an event to celebrate the achievement of 
learners for whom success in learning may be a ’first’, or the product 
of struggle, is often a moving, affirming and uplifting occasion for all 
present. Learners need encouragement, and it doesn’t get better 
than this. 



3. AREAS FOR DEVELOPMENT 

Staff Development 

3.10 Whilst we know what the framework for good practice is, many of the 
case studies in this volume highlight the fact that some tutors remain 
daunted by the prospect of assessment, and do not know how to 
work though all the stages listed in section 2. 

3.11 Many tutors are part-time and do not have a formal teacher-training 
qualification. For these tutors (and others), the emphasis on learning 
objectives and on-going assessment is new, and may represent a 
formidable challenge. 
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3.12 Examples of areas for staff development: 

© thinking of teaching and learning in terms of individuated objectives 

© clearly identifying specific learning objectives 

© recognising not only individual but also group learning objectives 

© distinguishing learning and learning progress, on the one hand, and 
teaching, and course evaluation on the other 

© ability both to retain a clear structure for a course or learning 
programme whilst also allowing for differentiated learning and 
different learning styles 

© recognising the importance of learner self-assessment, and 
encouraging the learner critically to reflect upon their progress 

© providing supportive and discerning critical feedback 

3.13 The new FE Teaching Qualifications Framework comes into force from 
September 2001 . This will apply to many, if not all tutors, who are 
delivering non-accredited provision, including: teachers of basic skills; 
new unqualified staff; part-time, fractional and full-time staff. Staff 
development for tutors will therefore take account of the FENTO and 
PAULO lecturer standards, and the differing levels of demand they make 
upon tutors (depending upon status). 20 

3.14 Priorities for staff development include: 



• identifying aims and objectives 

• negotiating learning objectives 

• facilitating learner self-assessment 

• providing detailed and discerning formative assessment 

• describing learning outcomes 



Managing Change 

3.15 The implementation of change cannot be achieved either unilaterally 
or in a hurry. 

3.16 In several of the case studies, development of a learning outcomes 
strategy evolved over an extended period, building on successive 
models. It is not useful to think of a single blue-print that will fit 
effortlessly the circumstances of any one organisation. 





3.17 Many if not all the success stories have in common the element of 
involving people at many levels in the organisation, including, in 
particular, learners and tutors. 

3.18 NIACE is already embarked on research into learner's perceptions' 21 
Efforts made to find out what learners themselves think of efforts to 
validate their learning are revealing (see section 4 below). One point 
deserves mention here: we ought not to anticipate in advance of asking 
them what learners themselves want; nor should we generalise across 
all learners and all cohorts. Different learners, unsurprisingly, want 
different things, and any system of validating learning that rides 
roughshod over such differences is liable to run into trouble. 

3.19 For many tutors the framework for good practice recommended will 
cause a culture shock; it is not a minor adjustment but a radical 
departure from past practice. 

3.20 Training and development has proved most successful when tutors are 
able to explore their own best methods for validating learning. Not only 
can this produce imaginative and valuable results, but tutors welcome 
the opportunity for participation that is more than merely nominal. 

3.21 One opportunity for staff development is presented at the internal 
moderation process. Tutors have opportunities to reflect on their own 
methods of assessment, and to learn from the good (and bad) practice of 
others. 

3.22 In summary, a process for introducing a system for validating learning 
should seek to be: 



• participatory (not a unilateral external imposition) 

• evolutionary (not introduced all at once) 

• inclusive of the views of learners 

• inclusive of the views of tutors 



Quality Assurance 

3.23 For many organisations, the validation of learning outcomes is above all 
a means of assuring quality in teaching and learning. 

3.24 Assuring quality in respect of learning may take the form of verifying: 

(i) learning outcomes; (ii) distance travelled; (iii) added value, (i) requires 
that the tutor verify that the learner has achieved their learning 
objectives, (ii) requires that the tutor has identified the learner’s starting 
and end points, and the dimension along which the learner moves 
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between the two; (iii) requires knowledge of the learner's starting and 
end points, and a comparison between this learner and the norm for 
learners of this type in a relevantly similar area of the curriculum. Each 
of these three tasks raises difficulties which are noted in section 4. 

3.25 A priority for quality assurance is to establish a common understanding 
of such key terms as mentioned in 3.24 above. Opportunities for 
assuring quality arise at several stages: 

® course review by head of department/curriculum manager 

® programme review of, e.g., retention, achievement and progression 

® intra-college comparison of year-on-year achievement rates of 
learners 22 

® internal (and external) moderation 

3.26 Quality assurance can take place with reference to external or national 
standards, including such documents as: 

» The Common Inspection Framework 23 

® FENTO Standards for Teaching and Supporting Learning 24 

® Raising Standards in Post-16 Learning 25 

4. WHAT WE DON'T KNOW 
Itemising the Unknowns 

4.1 Reading through the case studies, and listening to the concerns 
expressed by practitioners up and down the country, the questions 
above all which remain without adequate answers are these: 



1 . How do learners benefit from the process of recording and 
validating achievement? 

2. What body of evidence will satisfy inspection requirements? 

3. How do we measure learning (especially 'soft' outcomes)? 

4. How do we compare achievement rates across providers whose 
standards may differ? 
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Preliminary Points 



4.2 The four questions focus on, respectively: learners', evidence ; 
measurement ; and benchmarks. 

4.3 The questions have generated much anxiety, and for one or more of 
three quite distinct reasons: 

© In some cases no one yet has the knowledge the questions ask for 

© In some cases practitioners remain unaware of what is already 
known. 

® In many cases what is of concern are problems of administration 
and implementation. 

4.4 Sections 2 and 3 of this chapter, in common with the rest of this 
publication, represent an attempt to make headway with the second and 
third of these points; what follows is an attempt to explain what lies 
behind the first. 

4.5 We are making more headway with some questions than with others. 

For example, we already possess some information about students' 
perceptions (see below). But on the fourth bulleted question, for 
example, there is, to say the least, some way to go before we can 
identify reliable benchmarks so as to enable valid comparisons of 
achievement across different providers and curriculum areas. 

How do learners benefit from the process of recording and validating 
achievement? 

4.6 We do not know anything like as much as we should about what 
learners themselves think of procedures for recognising and validating 
their learning. 

4.7 NIACE is currently pursuing extensive research into learners' 
perceptions, and their conclusions will provide an important piece 
of evidence regarding the nature and desirability of the process as 
it is now conceived. 

4.8 The Mary Ward Centre has held a series of focus groups to research 
students' perceptions of their learning in uncertificated courses 26 This 
research, confirms conclusions reached by others; that, for example, 
we need to know more about: 

(a) the nature and value of unanticipated outcomes and the wider 
benefits of learning; 27 

(b) the perceptions of learners in response to formative and summative 
methods of recording learning; 

(c) what learners themselves think about attempts to validate (i.e. not 
merely record) their learning. 
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4.9 Work also remains to be done to determine the role for learners' 

perceptions in a record of assessment. The problem is nicely captured 
by a manager quoted in the City Lit case study, who emphasises the 
importance of: 

separating the value of evaluation to students and the necessity for monitoring effort by 
the City Lit. It is difficult to accomplish both tasks without destructive conflict and losing 
sight of the ultimate aim of the City Lit - student learning. If the evaluation process is 
basically student-centred, assessment is too subjective. If evaluation is basically tutor- 
centred, the City Lit drifts towards grading students which can be counter productive. 
On the other hand, a self-assessment needs to be validated by the tutor to get the full 
picture of progress. All this emphasises the importance of thorough training for tutors 
in implementing the system 28 . 



What body of evidence woll satisfy inspection requirements? 

4.10 What evidence exactly, and how much of it is required? No one yet has 
all the answers, and the stipulation in the consultation document for the 
Common Inspection Framework that 'evidence gathered is sufficient’, 29 
hardly tells us all we need to know. There are important gaps in this 
area, but also pockets of knowledge. I indicate the extent of both in what 
follows. 

4.11 Evidence of assessment is either quantitative or qualitative. It is 
quantitative whenever it takes the form of statistical information; 
otherwise it is qualitative, and might include exhibitions, displays and 
demonstrations of learners work. 

4.12 Quantitative evidence is an ineliminable part of the process of validating 
learning. All providers will be required to provide statistical information 
about each student on each course, and this will require the supply 

of evidence relating to attendance, retention and progression. 

4.13 In addition inspection will look at the extent to which ‘trends in 
performance over time show continuous improvement or the 
maintenance of very high standards' , 30 Evidence might take the 
form of information about the achievement of learners at any one 
institution pursuing a specified programme, or programmes, as 
compared with similar cohorts in previous years. 

4.14 One potential source of statistical information lies in determining the 
percentage of learning outcomes achieved by, eg. an individual learner, 
a class, learners in a curriculum area, and so on. 

4.15 An important unresolved question is the place of qualitative evidence 
in the record of assessment. Qualitative evidence is thought by some 
to provide a more meaningful record - for both tutors and learners - 
than that supplied by quantitative evidence alone. 
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4.16 Qualitative evidence may be drawn from audio, video or photographic 
evidence of learners applying new knowledge and skills outside a 
designated learning environment, including, for example, the home, 
the community or the work-place. 

4.17 One option worth exploring would require the production of qualitative 
evidence from a sample of students only, whilst the quantitative data 
continues to be drawn from evidence relating to all learners. 

4.18 A recurrent message from the case studies is that the procedure for 
collecting and recording evidence should be as simple as possible. 

We know that complication and repetition turns tutors and learners off, 
and once this happens, the quality of returns becomes poor. 

4.19 A key question not explored here is how institutions, and their MIS 
systems, will collect, store and extrapolate from evidence of learning, 
and learners’ progression. 



How do we measure learning (especially 'soft* outcomes}? 

4.20 It is mistake to suppose that nothing is known about the measurement 
of learning. Measurement requires that we know (i) the starting point; (ii) 
the end point; (iii) the dimension along which the learner is progressing, 
(i) requires information gleaned from initial assessment, or from a 
discussion about learning objectives; (ii) requires information gleaned 
from either formative or summative assessment (or both); (iii) requires 
clarity as to the course aims and objectives. In some cases it is both 
possible and desirable to acquire information of all these kinds. 

4.21 It is not always desirable or possible to measure learning in the sense 
of 4.19 above. Initial assessment is not always desirable, as in the case 
of short courses designed for hesitant or marginalised learners for whom 
the prospect of assessment may act as a deterrent. And it isn't always 
possible, as when we consider the case of soft outcomes. 

4.22 Soft outcomes are 'soft' in virtue of being difficult to quantify. 'Soft 1 is an 
unfortunate epithet, implying as it does something which is not only other 
than but also somehow inferior to 'hard'. But soft outcomes - building 
self-confidence for example - are of the first importance to many learners 
pursuing progression provision. 

4.23 Whether or not measurement of soft learning progress is possible in 
principle, it is often very difficult in practice. We can identify the 
promotion of self-confidence as a goal, and go some way towards 
forming judgements about learners’ self-confidence at the end of a 
course. But there is nevertheless plenty of hard work to be done on 
specifying exactly what it is that we are assessing, and exactly how we 
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are in a position to verify that a learner's self-confidence is as she or her 
tutor says it is. 

4.24 Measurement and verification: 'measurement' requires the specification 
of the distance travelled between the beginning and end of the course. 
'Verification' requires that we determine whether a learner has achieved 
the agreed learning objectives. With this distinction in mind, we might 
ask: is the measurement of learning always a desirable objective? There 
is a case for saying that what is important is that we should know what 
learners can do or understand at the end of their course, something 
which requires the verification of learning outcomes but not the 
measurement of distance travelled. If, on the other hand, measurement 
remains a sought after goal, there is already a great deal of work on this 
subject to be consulted, both in the area of basic skills, and in the area of 
ESF-funded projects, which DfEE commissioned studies are now 
exploring for evidence relating to measurement of non-accredited 
distance travelled. 

4.25 What follows from the draft report on performance indicators is decidedly 
vague, but it nevertheless gives a clear idea of the direction we are 
moving in: 

* In the interim [April 2001 -August 2002], it will only be possible for LSC to operate 
with a crude indicator of achievement rates across all post-16 provision, based 

upon the achievement of a ‘positive outcome ' however such a concept is defined 

31 

across the sub-sectors of provision.' 

♦ The second stage of development [ie. after August 2002) will be to develop ways 
of measuring distance-travelled and value-added to contribute to the achievement 

of a level playing field between different types of provision/client group. ACE 

32 

recording of learning outcomes may provide a starling point.' 



• 'ES is looking at distance-travelled in connection with New Deal. Likewise DfEE is 
exploring the issue in relation to the Connexions Learning Gateway and schools, 

has recently completed research on current practice within ESF-funded projects, 

33 

and is seeking to develop a Post-16 measure of value-added.’ 

• ‘It will be for LSC to consider how to take this initial development work forward to 
achieve a robust performance measure of distance travelled and value added in 
the long tern?.’ 34 

Question 4: How do we compare achievement rates across providers? 

4.26 We know quite a lot about how to identify and verify learning, and we 
know something about its measurement. But we do not know very much 
at all about what is perhaps the most important question of all: how are 
we to compare the standards of assessment applied by one provider 
with those employed by another? 

4.27 If college A sets high standards which 50% of learners meet, whilst 
college B sets low standards which 95% of learners meet, the figures by 
themselves not only tell us very little, they are positively misleading. This 
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would occur, for example, if college A defined achievement for an 
individual learner as 'meeting at least 65% of the agreed learning 
objectives', whilst college B required only a success rate of 50% or 
above. The figures would be equally misleading if, notwithstanding that 
both A and B define success as meeting at least 65% of the agreed 
objectives, the standard for reaching any one objective is much higher in 
the case of A than in the case of B. 

4.28 How we identify reliable benchmarks is one of the most important and 
unresolved questions that we face in the area of progression provision. 
We can provide the evidence, but, without the means of providing 
reliable benchmarks, the evidence by itself will reveal little in the way of 
meaningful comparative information about cross provider performance 
and standards. 

4.29 From the Draft Final Report on Interim Provider Performance Indicators: 

" Because of the variation in types of outcome, achievement levels will not be 

directly comparable across Post-16 provision. However, as now, LSC will be able to 
compare providers within the same area of provision producing like for like 
outcomes.' 35 



5. CONCLUSIONS AND EMERGING ISSUES 

5.1 We now know a great deal about the recognition and validation of adult 
learning. The framework for good practice is clear, and there is 
agreement as to what the key elements in this framework are. Above all, 
there is a need for clarity in respect of learning objectives, and a 
recognition that assessment, which is at the heart of the entire process, 
should make reference to the objectives agreed with learners. 

5.2 We know what the framework is, but four fundamental questions remain 
unsolved: 



Questions for Research 

® how we measure (as distinct from verify) non-accredited learning 

• how we take account of soft outcomes 

® how we devise benchmarks to allow for reliable comparisons 
between providers 

e how learners perceive attempts to record and validate what 
they are doing 
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5.3 These questions are now the subject of ongoing research, 36 and it will be 
important also to take account of research effort in such closely related 
areas as: 

« Basic Skills 37 

® Widening Participation 38 

® Non-schedule 2 pilot projects 39 

and also to take account of research into such topics as: 

« Distance Travelled 40 
® Added Value 41 
o Soft Outcomes 42 

5.4 Meanwhile there remains the important task of consolidating what 
we do know, for there are many who are as yet unfamiliar with, even 
daunted by talk of assessment and learning outcomes. There are, in 
addition, many difficulties that arise only when we seek to apply the 
framework in the particular circumstances that providers face. Our first 
task, then, is to share with practitioners the body of knowledge and good 
practice we already possess in abundance. At the same time, we should 
not forget one of the most valuable lessons of our case studies: that the 
introduction of a system for validating achievement should be both 
participatory and evolutionary, and should, above all, aim to be 
inclusive of the views of learners themselves. 
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APPENDIX 1 



COMMON INSPECTION FRAMEWORK: KEY QUESTIONS 

Inspection will aim to answer the following overarching question: 

How effective and efficient is the provision of education andl training in 
meeting the needs of learners, and why? 

The following key questions will guide inspection: 



ACHIEVEMENT AND-SIAN QARQSL. 



•k Mow-wetl-do-leamers-aehieve? 



THEOWAtlTY-OF EDUCATION AND TRAINING 



2\ Howeffective-are teaching, training anti learning?" 



3. How are acFfievemerif and leanrimg" affectecfby resources? 



4. How effective are tjie_assessment jand mon]toj'ing_ofJearning? 



5, Hcus/jweJixiathejjcogfatnjpes^nid^cuJtses meetliie Deeds and _ 

interests DfJearnecs 2 



& How well-are learners guided and supported? 



tEABERSHtP-AND-MANAGEMENT- 



T. How effective ‘a re leadership anti management in raising 

ffcRievehriferit Shti supporting SIITeSrheFs? 
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NOTES 



1 Thanks are due to members of the Learning Outcomes Group, who. together with Anna 
Reisenberger, made insightful remarks about an earlier draft. 

2 The Common Inspection Framework for Inspecting Post-16 Education and Training , Adult 
Learning Inspectorate and the Office for Standards in Education, February 2001 . See 
Appendix 1 . 

3 Re-reading material from the early nineties one is repeatedly struck by the similarity 
between what was then regarded as good practice and the messages emerging from 
case studies in this volume. 

4 See for example, Donley, J., and Napper, R. t Assessment Matters in Adult Learning , 
Oxfordshire County Council and NIACE, 1999. 

5 See, for example, Daines, J., ‘Learning Outcomes in the WEA’, in Adults Learning 
September 1994. 

6 See, for example, Hayes, A., et. al. , Outcomes of Adult Learning, FEDA, 2000. 

7 CIF, p. 8. 

8 CIF, p. 10. 

9 The Inclusive Learning Initiative provides guidance on these matters. 

10 ClF, p. 8. 

1) CIF, p. 7. 

12 CIF, p. 8. 

13 CIF, p. 10. 

14 CIF, p. 12. 

1 5 Turner, C., Measure of achievement and eligibility criteria for funding non-accredited 
learning under the Learning and Skills Council, unpublished, NIACE, October 2000. This 
was intended merely to indicate what the evidential requirement might look like in some 
cases; it was not intended as either definitive or comprehensive, 

16 CIF, p. 10. 

17 ibid. 

18 See the sixth of the LOG key principles ('Bureacracy’): The bureaucratic underpinning of 
any system of recording learning and collecting evidence is proportionate to the 
institution s resources and not unduly intrusive of the learning process. See also the Mary 
Ward case study in relation to the moderation process. 

19 The first of the six key principles, under the name of ‘The Learner’, states that: the 
instrument developed to measure learning is appropriate to the learner’s needs and 
reflects good equal opportunities and inclusive learning practice. See p.11 this volume. 

20 See for example: Standards for Teaching and Supporting Learning, FENTO, January 
1999. 

2) See the chapter in this volume and their forthcoming publication. 

22 Comparisons across curricula and providers raise special difficulties which I discuss in 
section 4 below. 

23 The Common Inspection Framework, ALI and OFSTED, 2001. 
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BEST COPY AVAILABLE 



24 FENTO Standards for Teaching and Supporting Learning, FENTO, 1999. 

25 Raising Standards in Post-16 Learning, DfEE, 2000. 

26 Janssen, O., Students ' Perceptions of Learning in Uncertificated Courses , unpublished, 
February 2000. For a summary see Janssen, O., The Students' Voice’ in Adults 
Learning, October 2000. 

27 The good news is that there is now a research centre at the Institute of Education 
dedicated to pursuing research into this area. 

28 p-16 in this volume. 

29 Inspecting Post-16 Education and Training - Informal Consultation on the Common 
Inspection Framework, DfEE, April 2000. 

30 CIF, p. 7. 

3 3 Wilson, B., and Turner, B., (for the LSC), Draft Final Report From the ‘Interim Prowc/er 
Performance Indicators For Use By The LSC ; Project, November 2000. (‘Interim': April 
2000 - August 2002). Emphasis added. 

32 ibid. Emphasis added. (ACE: Adult and Community Education) 

33 ibid. (ES: Employment Service) 

34 ibid. Emphasis added. 

35 Wilson, B., and Turner, P., op. cit. 

36 A publication from NIACE and the Agency is due in October. 

37 See, for example, Grief, S., Recognising and Validating Learning Outcomes and 
Achievement in Non-Accredited Basic Skills and ESOL, The Learning and Skills 
Development Agency, forthcoming. 

38 See, for example, Taylor, S., Back on Track: Successful Learning Provision for 
Disaffected Young People , Research Report, Learning and Skills Development Agency, 
December 2000. 

39 See, for example, Greenwood, M., and Merton, A., An Evaluation of Non-Schedule 2 Pilot 
Projects , Learning and Skills Development Agency and NIACE. 

40 See, for example Dewson, S., et. al. , Measuring Soft Outcomes and Distance Travelled , 
Research Report 219, DfEE, 2000. 

4 1 See, for example, Clark, R., Developing Added Value Measures , AoC, 2000. 

42 See, for example, Dewson, S. et. al. , op. cit. 
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